Utilizing eye tracking to reduce power consumption involved in measuring affective response

ABSTRACT

Systems and methods that enable a reduction of the power consumption involved in measuring a user&#39;s affective response to content. The reduction in power consumption is achieved by utilizing eye tracking to determine when a user is paying attention to content, and accordingly setting a mode of operation of a device that measures the user. Thus, by using different modes of operation, which are characterized by different energy consumption rates, the total power consumption of the device may be reduced, without loss of relevant measurements.

BACKGROUND

Through numerous interactions with computer systems in their day-to-daylives, users are exposed to a wide array of content, such communications(e.g., video, instant messaging, or interacting with a virtual agent)and various forms of digital media (e.g., internet sites, televisionshows, movies, augmented reality, and/or interactive computer games).Throughout these many interactions, affective computing systems canmeasure a user's affective response to the content and analyze theinformation. This analysis may provide insight that can be used toimprove the current and/or future content presented to the user. Forexample, a computer game may be able to recognize if the difficultylevel of the game is appropriate for the user, and adapt the gameaccordingly. In another example, a television may learn what programs auser is likely to enjoy from monitoring the user's reaction topreviously shown programs. The television may then be able to select anddeliver more appropriate content to the user, with essentially no needfor the user to actively choose or intervene.

Advances in human-computer interaction have led to the development ofmultiple devices that may be utilized to measure a person's affectiveresponse. For example, systems like Microsoft Kinect™ that involveinexpensive cameras and advanced analysis methods are able to track auser's movement, allowing the user to interact with computers usinggestures. Similarly, eye tracking is another technology that is findingits way into more and more computing. Tracking a user's gaze with one ormore cameras enables computer systems to detect what content or objectsthe user is paying attention to. Other technologies found in anincreasing number of consumer applications include various sensors thatcan measure physiological signals such as heart rate, blood-volumepulse, galvanic skin response (GSR), skin temperature, respiration, orbrainwave activity such as electroencephalography (EEG).

Often, the devices used to measure users are mobile and/or wireless(e.g., bracelets with GSR sensors, cameras, headsets with EEG sensors orimplants); thus, they tend to required batteries for power. However,taking the multitude of measurements of a user's affective response,which affective computing application often demands, may consume a lotof power that may drain the sensors' batteries. Unfortunately, the needto supply sufficient power for operating sensors and/or analyzing thedata they produce is currently met by unsatisfactory means. For example,the size of the battery powering a sensor may be increased, but this isundesirable since it makes the mobile devices more cumbersome and lesscomfortable for the user. Another inconvenient possibility is torecharge power sources more often to meet the power demands. Yet anotherpossibility is to measure users less or decrease the quality of themeasurements, thus reducing the sensors' power demands. However, thistoo is problematic, since it means that the computer systems may reducethe quality of service provided.

Thus, there is a need to reduce the power consumption of devices used tomeasure affective response, and to do so without severelyinconveniencing users or dramatically reducing the quality of dataprovided by the devices.

BRIEF SUMMARY

Some aspects of this disclosure involve systems and/or methods thatenable a reduction of the power consumption involved in measuring auser's affective response to content. The reduction in power consumptionis achieved by utilizing eye tracking to determine when a user is payingattention to content, and accordingly setting a mode of operation of adevice that measures the user. Thus, by using different modes ofoperation, which are characterized by different energy consumptionrates, the total power consumption of the device may be reduced, withoutloss of relevant measurements.

Some embodiments involve an eye tracking module and a device usedmeasure affective response. For example, the eye tracking module mayinclude an image capturing device (e.g., a video camera) that detects adirection of a user's gaze and/or determines objects the user is lookingat. The device may be a physiological sensor, such as a headset withembedded EEG electrodes. Given a request to measure the user's affectiveresponse to a segment of content, the system may utilize eye trackingdata of the user in order to determine a mode of operation for thedevice. For example, a television system with affective computingcapabilities may issue a request to measure a user's affective responseto a certain segment of video with the EEG headset. However, if the eyetracking data determines that the user is not looking at the screen atthe time, the system may elect not to measure the user with the EEGheadset; the EEG measurements that would have been produced would nothave likely reflected the user's affective response to the segment ofvideo.

Some embodiments include a controller that includes a processor. Theprocessor may utilize analysis of eye tracking data in order todetermine if a gaze-based attention level reaches a predeterminedthreshold. For example, the processor may determine whether the user islooking in the right direction, gazing at the content for a sufficientlylong time, and/or whether a pattern of the user's eye movement indicatessufficient attentiveness. Optionally, based on whether or not thepredetermined threshold is reached, the processor may select anappropriate mode of operation for a device used to measure the user.

In some embodiments, the controller includes a first interfaceconfigured to receive tags corresponding to segments of content. Thetags indicate to the controller durations in which to measure theaffective response of the user to the segments by operating the devicein a first mode of operation. The controller also includes a secondinterface configured to receive eye tracking data of the user taken intemporal vicinity of the durations indicated by the tags. Thesemeasurements are provided to the processor that evaluates whether agaze-based attention level of the user reaches a predeterminedthreshold. Based on the results of the evaluation, the processor selectsa mode of operation for the device. The processor may select the mode ofoperation from a set that includes first and second modes of operation.The power consumption of the device when operating in the first mode ofoperation is significantly higher than the power consumption of thedevice when operating in the second mode of operation. Thus, the systemmay reduce its power consumption for measuring affective response withthe device when it is determined, according to the eye tracking data,that the user is not paying sufficient attention to the content.

BRIEF DESCRIPTION OF THE DRAWINGS

The embodiments are herein described, by way of example only, withreference to the accompanying drawings. In the drawings:

FIG. 1 illustrates a system that includes a controller configured toutilize eye tracking data to reduce power consumption of a deviceoperative to measure affective response;

FIG. 2 illustrates one embodiment of a method to utilize eye trackingdata to reduce power consumption of a device used to measure affectiveresponse;

FIG. 3 illustrates media content having four segments and three tags;

FIG. 4 illustrates one embodiment that includes a module for training amodel for a measurement Emotional Response Predictor (measurement ERP);

FIG. 5 illustrates an interactive computer game that generates segmentsand tags;

FIG. 6 illustrates a system that includes a controller that utilizesmeasurements of a sensor to reduce power consumption of a deviceoperative to measure affective response;

FIG. 7 illustrates one embodiment of a method for utilizing a sensor toreduce power consumption of a device used to measure affective response;

FIG. 8 illustrates one embodiment of a system configured to reduce powerconsumption by overriding an instruction to measure conveyed by a tag;

FIG. 9 illustrates one embodiment of a method for reducing powerconsumption by overriding an instruction to measure conveyed by a tag;

FIG. 10 illustrates one embodiment of a system configured to reducepower consumption by utilizing eye tracking to override instructionsconveyed by tags;

FIG. 11 illustrates one embodiment of a method for reducing powerconsumption by utilizing eye tracking to override instructions conveyedby tags;

FIG. 12 illustrates one embodiment of a system configured to reducepower consumption according to analysis of content;

FIG. 13 illustrates one embodiment of an apparatus configured to reducepower consumed measuring affective response;

FIG. 14 illustrates one embodiment of a method for reducing powerconsumed measuring affective response;

FIG. 15 illustrates one embodiment of a computer game that outputssegments of content and information about the expected emotionalresponse to the segments;

FIG. 16 illustrates one embodiment of a system configured to reducecomputational load involved in processing measurements of affectiveresponse of a user to content;

FIG. 17 illustrates one embodiment of a method for reducingcomputational load of processing measurements of affective response of auser to content;

FIG. 18 illustrates one embodiment of a system configured to usesemantic analysis to measure affective response at varying measuringrates;

FIG. 19 illustrates one embodiment of a system configured to enable adevice to measure affective response at varying measuring rates based onsemantic analysis of content;

FIG. 20 illustrates one embodiment of a method for utilizing semanticanalysis to set a measuring rate of a device that measures affectiveresponse;

FIG. 21 illustrates one embodiment of a computer game that outputssegments of content and context information about the segments;

FIG. 22 illustrates one embodiment of a system configured to utilizesemantic analysis to set a processing level for processing affectiveresponse measurements;

FIG. 23 illustrates one embodiment of a measurement ERP trainer fortraining an emotional response model;

FIG. 24 illustrates one embodiment of a measurement Emotional ResponsePredictor (measurement ERP);

FIG. 25 illustrates one embodiment of a method that utilizes semanticanalysis to set a processing level of a processor processing affectiveresponse measurements; and

FIG. 26 illustrates one embodiment of system configured to utilizesemantic analysis to set processing level for affective responsemeasurements.

DETAILED DESCRIPTION

Affective response measurements of a user refer to measurements ofphysiological signals of the user and/or behavioral measurements of theuser, which may be raw measurement values and/or processed measurementvalues (e.g., resulting from filtration, calibration, and/or featureextraction). Measuring affective response may be done utilizing variousexisting, and/or yet to be invented, devices such as sensors, which canbe attached to a user's body, clothing (such as gloves, shirts,helmets), implanted in the user's body, and/or be placed remotely fromthe user's body.

Herein, “affect” and “affective response” refer to physiological and/orbehavioral manifestation of an entity's emotional state. The terms“affective response/state” and “emotional response/state” may be usedherein interchangeably. However, affective response typically refers tovalues obtained from measurements and/or observations of an entity,while emotional responses are typically predicted from models orreported by the entity feeling the emotions. In addition, the terms“state” and “response”, when used in phrases such as “emotionalstate”/“emotional response” and “affective state”/“affective response”,may be used herein interchangeably; however, in the way the terms aretypically used, the term “state” is used to designate a condition inwhich a user is in, and the term “response” is used to describe anexpression of the user due to the condition the user is in or due to achange in the condition the user is in. For example, according to howterms are typically used in this document, it by be said that a person'semotional state (or emotional response) is predicted based onmeasurements of the person's affective response.

Phrases like “an affective response of a user to content”, or “a user'saffective response to content”, or “a user's affective response to beingexposed to content” refer to the physiological and/or behavioralmanifestations of an entity's emotional response to the content due toconsuming the content with one or more of the senses (e.g., by seeingit, hearing it, feeling it). Optionally, the affective response of auser to content is due to a change in the emotional state of the userdue to the user being exposed to the content.

In some embodiments, content refers to information (e.g., data and/ormedia) which a user may consume, such as communications (e.g.,conversations, messages), video clips, movies, music, augmented realityobjects, virtual reality objects, and/or computers games. A segment ofcontent may include one or more portions of content that the user may beexposed to over a period of time, and/or portions of multiple contentitems.

In some embodiment, content may comprise data representing text. Herein,data representing text may be data that includes text and/or data thatmay be converted to text. In one example, a video may be considered datarepresenting text, since the video may be converted to a transcriptwhich describes a dialogue in the video and/or events occurring in thevideo. In another example, a voice conversation may be considered datarepresenting text, since the conversation may be transcribed. In yetanother example, a web page may be considered data representing textsince text may be extracted from certain portions of the code and/ordata objects used to generate the web page (e.g., text may be extractedfrom XML code or scripts utilized to generate the web page).

FIG. 1 illustrates a controller 258 configured to utilize eye trackingdata to reduce power consumption of a device 252 operative to measureaffective response. Optionally, the eye tracking data is acquiredutilizing an eye tracker 251, and the eye tracking data is data of auser 114. Optionally, the device 252 measures affective response of theuser 114. The controller 258 includes at least a first interface 254, asecond interface 255, and a processor 256.

In some embodiments, the first interface 254 receives tags correspondingto segments of content to which the user 114 is exposed. Optionally, thefirst interface 254 is implemented, at least in part, in software.Optionally, the software runs on the processor 256. In one embodiment,the first interface 254 is configured to receive at least a first tag292 a and a second tag 292 b: The first tag 292 a indicates a firstduration during which to measure affective response to a first segmentof content by operating the device 252 in a first mode of operation. Thesecond tag 292 b indicates a second duration during which to measureaffective response to a second segment of content by operating thedevice 252 in the first mode of operation.

Herein, a first segment of content and a second segment of content arenot necessarily segments of the same content item. For example, thesegment may be a clip from a first movie, and the segment may be a clipof a second, different, movie. In another example, the first segment maybe a portion of a video game play, and the second segment may be from adifferent type of content, such as a web page. Optionally, the first andsecond segments may be different segments of the same content item. Forexample, the first segment and the second segment may correspond tofirst and second scenes in the same movie.

A tag, such as the tags 292 a and/or 292 b, may comprise data that maybe used to convey information regarding a segment of content. Theinformation can be related to various aspects, such as details about thesegment of content and/or information that may be used by systemelements in order to perform actions related to the segment of content.

FIG. 3 illustrates a media content having four segments (289 a, 289 b,289 c, 289 d), and three tags (283 a, 283 b, 283 c). The tags areindicative of durations in which to measure the affective response ofthe user to the segments by operating a device, such as the device 252,in a first mode of operation. In some embodiments, some segments may nothave a tag associated with them (e.g., segment 289 c). Optionally, asegment may have more than one tag associated with it, and/or a tag maybe associated with more than one segment.

In one embodiment, a tag, such as the tags 292 a and/or 292 b, mayindicate starting times, ending times, and/or lengths of time for takingmeasurements of the affective response of the user 114 with a device,such as the device 252. Optionally, a tag may indicate a durationrelative to when the tag was and/or to when a segment is expected to beexposed to the user. For example, the fact that the tag is received,indicates to start measuring with the device or to start measuring aftera certain predetermined time. In another example, the tag may providethe length of the duration during which the device should measure.Optionally, a tag may define an absolute time and/or duration (e.g.,from 9:42:12 until 9:42:56 that same day).

It is noted that phrases like “to indicate something” or “indicative ofsomething” are to be interpreted as directly and/or indirectly signalingthe something. For example, a phrase like “the tag indicates a mode ofoperation”, means that the tag directly or indirectly signals the modeof operation.

The second interface 255 receives eye tracking data of the user 114taken by an eye tracker 251 in temporal vicinity of durations indicatedby the tags. Optionally, the second interface 255 is implemented, atleast in part, in software. Optionally, the software runs on theprocessor 256. In one embodiment, the second interface 255 is configuredto receive at least first and second eye tracking data of the user 114:The first eye tracking data is acquired in temporal vicinity of thefirst duration indicated by the first tag 292 a. The second eye trackingdata is acquired in temporal vicinity of the second duration indicatedby the second tag 292 b.

Temporal vicinity refers to closeness in time. For example, two eventsthat occur in temporal vicinity, occur at times close to each other. Thedifference in the time between the occurrences required for the twoevents to be considered in temporal vicinity depends on the context ofthe events and/or the duration of the events. In one example, statingthat eye tracking data is acquired in temporal vicinity of a durationmeans that the tracking of the user that produced the eye tracking dataoccurred essentially during the duration. For example, the time thetracking took place might have been up to two seconds before the startof the duration, during the duration, and/or up to two seconds after theduration. In another example, stating that a measurement of the user istaken in temporal vicinity of an exposure to a segment of content (e.g.,a video clip) may mean that the measurement is taken slightly before theclip starts (e.g., less than a minute before) and/or during thepresentation of the clip. In some cases, it may mean that themeasurement is taken, at least in part, during a period that extendsshortly after the exposure to the segment of content (e.g., a fewseconds after the clip ends).

In one embodiment, a duration, indicated by a tag is a time periodduring which a user is to be measured by a device, such as the device252. Optionally, the duration may overlap with a period of time in whichthe user 114 is exposed a segment of content corresponding to the tag.Optionally, the duration may span part of the period during which theuser is exposed to the segment of content corresponding to the tag.Alternatively or additionally, the duration may span a period that ends,and possibly even starts, after the user 114 is exposed to segment ofcontent corresponding to the tag. In one example, a segment of contentmay be short (e.g., a couple of seconds long), while the device used tomeasure the user may be a device with a relatively slow changing signalsuch as GSR. In this case, the affective response may be reflected inthe measurement values of the device substantially after the segment ofcontent was already exposed to the user 114 (e.g., the affectiveresponse may be reflected in measurements ten seconds after theexposure).

In one embodiment, the first and second eye tracking data of the user114 acquired in temporal vicinity of the first and second durations areprocessed prior to being evaluated by the processor 256. Optionally, theeye tracking data may undergo filtration, normalization, and/or featureextraction. Optionally, processing the first and second eye trackingdata may involve analysis of the eye tracking data to determine fordifferent times directions of gaze of the user and/or objects the userwas looking at. Optionally, at least some of the processing of the eyetracking data is performed by the processor 256. Alternatively oradditionally, at least some of the processing is performed by the secondinterface 255 and/or the eye tracker 251.

The processor 256 is configured to analyze the first and second eyetracking data, and to select a mode of operation for the device 252based on the evaluation. The processor 256 may compare the eye trackingdata of the user 114 and/or data derived from the eye tracking data to apredetermined threshold in order to determine whether the predeterminedthreshold is reached. Optionally, given a duration indicated by a tag,the processor 256 may assign an appropriate mode of operation based onwhether or not the eye tracking data, which correspond to the duration,reach the predetermined threshold.

In one embodiment, the processor 256 analyzes the first eye trackingdata and determines that a gaze-based attention level to the firstsegment reaches a first predetermined threshold. Optionally, based onthe fact that the first predetermined threshold is reached, theprocessor 256 selects the first mode of operation 287 a for operatingthe device 252, during the first duration indicated in the first tag 292a, to measure affective response of the user 114 to the first segment.Additionally or alternatively, the processor 256 analyzes the second eyetracking data and determines that a gaze-based attention level to thesecond segment does not reach a second predetermined threshold.Optionally, based on the fact that the second predetermined threshold isnot reached, the processor 256 selects a second mode of operation 287 bfor operating the device 252, during the second duration indicated inthe second tag 292 b, to measure affective response of the user 114 tothe second segment.

In one embodiment, the processor 256 utilizes a value representing agaze-based attention level from eye tracking data. Optionally, the valueis compared to a predetermined threshold in order to determine if thegaze-based attention level reaches the predetermined threshold.Optionally, at least part of the computation involved in computing thevalue representing the gaze-based attention level is performed by theprocessor 256. Additionally or alternatively, at least some of thecomputation involved in determining the value representing thegaze-based attention level is performed by other modules, such as theeye tracker 251 and/or a remote processor (e.g., a cloud-base service).Additional information regarding eye tracking and computing thegaze-based attention level is given below.

Herein, by stating that a value reaches a predetermined threshold, it ismeant that the value equals or exceeds the predetermined threshold. Inaddition, a predetermined threshold to which a value a measurement ofaffective response is compared, refers to a value of which there isprior knowledge. For example, the threshold value itself is known and/orcomputed prior to when the comparison is made. Additionally oralternatively, a predetermined threshold may utilize a threshold valuethat is computed according to logic (such as function) that is knownprior to when the comparison is made.

In one embodiment, the first predetermined threshold and the secondpredetermined threshold may represent different thresholds. For example,reaching the first predetermined threshold is based on achievingdifferent value than is required to pass the second predeterminedthreshold. Thus, there may be cases in which a certain value may reachthe first predetermined threshold, but not reach the secondpredetermined threshold. Alternatively, in another embodiment, the firstpredetermined threshold and the second predetermined threshold mayrepresent essentially the same predetermined threshold. Thus, in thisembodiment, a value that reaches the first predetermined threshold alsoreaches the second predetermined. Similarly, in this embodiment, a valuethat does not reach the first predetermined threshold also does notreach the second predetermined threshold.

In one embodiment, a predetermined threshold, such as the first orsecond predetermined thresholds, has a fixed value. For example, thepredetermined threshold may be 50%, representing that a user's eyetracking data should indicate that the user looked at the content for atleast half of the time. In another example, the predetermined thresholdmay be 7, and represent the fact that a user's gaze-based attentionlevel, as determined by a certain function that computes gazed-basedattention level from eye tracking data on a scale of 1 to 10, is 7 ormore.

In another embodiment, when a predetermined threshold is computedvarious factors may be taken into account, such as the length of theduration during which eye tracking data was acquired, the type ofcontent the use was exposed to, and/or characteristics of the userand/or the environment. Thus, based on one or more of these factors, thepredetermined threshold may represent lower or higher degrees ofrequired attention in order for the predetermined threshold to beconsidered reached. In one example, a longer duration in which eyetracking is acquired may lead to a decrease in a predetermined thresholdthat represents a percentage of time a user looks at the content; it maybe difficult in some situations to continually look at content (e.g.,when outside in the company of other people), so the threshold can berelaxed in such situations. In another example, the predeterminedthreshold used with a user that is a child may be lower than that of anadult, since for some children, it is more difficult to stayconcentrated for long. In another example, the predetermined thresholdused with content that is a commercial may be lower than thepredetermined threshold used for content that is a movie or televisionprogram, since users tend to pay less attention to commercials (thoughoften measurements in these situations are desired). In still anotherexample, the type of display on which content is consumed may influencethe predetermined threshold. For instance, a display that ishead-mounted (e.g., an augmented reality display) may have a different,possibly higher, predetermined threshold associated with it, compared toa display on a smartphone, since content on an augmented reality displaymay be clearer and easy to see if the user desires to look at it.

In some embodiments, a selected mode of operation has preset operationparameters (such as voltage, resolution, sampling rate). Alternativelyor additionally, the mode of operation may include certain operationparameters that are computed according to a received tag, measurementvalues of the device 252, and/or other data that is available at thetime of computation. Optionally, selecting a mode of operation mayinvolve computing certain parameters related to the mode of operation.Thus, for example, when the processor 256 selects the first mode 287 aor the second mode 287 b, this may involve computing certain parametersfor the modes of operation (e.g., duration of measurements, voltage of asensor, and/or a sensor sampling rate).

In some embodiments, power consumption of a device, such as the device252, while operating in the first mode of operation 287 a may besignificantly higher than its power consumption while operating in thesecond mode of operation 287 b. Consequently, by selecting for certaindurations, the second mode of operation 287 b for the device 252 tooperate in, instead of the intended first mode of operation 287 a, thetotal amount of energy consumed over time by the device 252 may bereduced. Additionally, the controller 258 may reduce the powerconsumption of other elements that participate in the processing of theaffective response measurements taken by the device 252, such as aprocessor that processes the data comprising the measurements of thedevice 252. Optionally, the power consumption of the device 252 whileoperating at the second mode of operation 287 b is essentially zero. Forexample, by essentially zero it is meant that the power consumption isat most 5% of the power consumption of the device 252 when operating atthe first mode of operation 287 a.

In one embodiment, two modes of operation may be considered differentmodes of operation, and be referred to as “a first mode of operation”and “a second mode of operation” in the claims, if the differencebetween power consumption of a device operating in each of the modes issignificant. If the difference in power consumption of the device whenoperating in the two modes is not significant, then the two modes arenot “a first mode of operation” and “a second mode of operation” as usedin the claims. In one example, a significant difference refers to adifference of at least 100% in the power consumption, e.g., the powerconsumption of the device 252 operating in the first mode of operation287 a is at least double the power consumption of the device 252operating in the second mode of operation 287 b. In another example, asignificant difference refers to a difference of at least tenfold in thepower consumption, e.g., the power consumption of the device 252operating in the first mode of operation 287 a is at least ten times thepower consumption of the device 252 operating in the second mode ofoperation 287 b. In yet another example, if when operating in the secondmode of operation 287 b, power consumption of the device 252 isessentially zero, while power consumption of the device 252 operating inthe first mode of operation 287 a is not essentially zero, then thedifference in power consumption of the two modes of operation isconsidered significant.

In one embodiment, a mode of operation is selected from a set thatincludes at least two modes of operation: a first mode of operation thatis a standby mode of operation, and a second mode of operation that is anormal mode of operation. In this example, as is the case with manydevices, when the device operates in the standby mode, it consumessignificantly less power compared to when it operates in the normal modeof operation.

In one embodiment, the mode of operation that is selected may determineparameters for operating, such as a resolution of images captured by adevice which is a camera. If the predetermined threshold is reached, afirst mode of operation in which the camera captures high-resolutionimages (e.g., 1920×1080 pixels) may be selected. In this case, usinghigh resolution images may enable better analysis of the user'saffective response, since it can enable better detection of gesturesand/or facial expressions. However, If the predetermined threshold isnot reached (indicating that the user is not paying sufficientattention), the processor may select a second mode of operation in whicha camera captures low-resolution images (e.g., 320×240 pixels), whichenables only basic image analysis. In some cases, different resolutionslead to different power consumptions for acquiring images. For instance,the lower resolution of the second mode of operation can cause a camerato utilize significantly less CMOS gates, compared to the number of CMOSgates it needs to utilize in the first mode of operation. Thus, in thisexample, the power consumption of the device operating in the first modeof operation is significantly higher than the power consumption of thedevice operating in the second mode of operation.

In another example, the device may be a battery operated headset thatmeasures brainwaves with EEG. If the threshold is reached, a first modeof operation in which the user's brainwaves are measured extensively(e.g., by measuring multiple bands of frequencies) may be selected. Forexample, measuring the user with the EEG may help determine to moreprecisely how the user felt towards elements in the content. However, ifthe predetermined threshold is not reached, the EEG measurements are notlikely to reflect the user's attitude towards the content, so a secondmode of operation may be selected in which the user is not measured atall, or measured rudimentary (e.g., by monitoring less bands offrequencies).

In one embodiment, a mode of operation for a device, such as the modes287 a and/or 287 b, may be implicitly or explicitly selected. Forexample, an implicit selection of the mode of operation may occur if thedevice 252 does not receive a signal that may indicate and/or influencea mode of operation at which it operates. By contrast, an explicitselection of the mode of operation may involve sending the device 252information that indicates which mode of operation to operate in and/orparameter values that determine how the device 252 is to operate.

A device, such as the device 252, may operate in different modes ofoperation, which can dramatically change its power consumption. Forexample, the controller can cause the device to change from an operablemode to a substantially inoperable mode (or standby mode), whichconsumes significantly less power, and vice versa. It is to be notedthat a term such as “significantly more” or “significantly higher”, whenreferring to power consumption of the device, refers to a differencegreater than a factor of two between the lower and higher values ofpower consumption being compared. To illustrate this point, consider adevice capable of operating in two modes of operation: A and B. When itis stated that the device consumes significantly more power whileoperating in mode of operation A compared to the power it consumes whileoperating in mode of operation B, it is meant that the energy consumedby the device, per unit of measurement time, while operating in mode ofoperation A, is at least double the energy consumed by the device, perunit of measurement time, while operating in mode of operation B.Similarly, a phrase like “significantly less”, when referring to powerconsumption of a device, refers to a factor of at least two between thehigher and lower values of power consumption.

It is to be noted that the phrase “per unit of measurement time” refersto a period of time in which the device may be operated and used tomeasure affective response. In this phrase, “measurement time” refers toa slot of time, during which the user may be measured with the device;however, in some cases, the system may elect not to measure the userwith the device (e.g., by keeping it off or in standby mode). In oneexample, a unit of measurement time may be one second or one minute.

The value of power consumption may be computed as the energy consumptionper unit of measurement time. The power consumption of a device may becomputed by dividing the total energy consumed by the device by theduration of the measurement time (e.g., expressed in seconds). Thisvalue serves as a normalized value that enables the comparison of powerconsumption of different devices, and/or of the same device whenutilized to measure affective response for varying durations and/orwhile operating in different modes of operation.

Therefore, when a device operating in mode of operation A is said toconsume significantly more power than when it operates in mode ofoperation B, it is meant that for equivalent measurement times, theenergy consumed by the device when operating in mode of operation A isat least double the energy consumed by the device when operating in modeof operation B.

In one embodiment, a unit of measurement time may refer to a contiguousperiod of time in which the device may operate. Thus, for example, if ina fixed period of time, such as one second, the device (or a componentof the device) may rapidly alternate between being turned on and off, sothat half of the time it is on and half of the time it is off, themeasurement time of the device is considered to be the full second.

FIG. 2 illustrates one embodiment of a method for utilizing eye trackingdata to reduce power consumption of a device used to measure affectiveresponse. The method includes the following steps:

In step 260, receiving a first tag that indicates a first durationduring which to measure affective response to a first segment of contentby operating the device in a first mode of operation.

In step 261, receiving a first indication, derived from first eyetracking data of a user acquired in temporal vicinity of the firstduration, indicating that a gaze-based attention level to the firstsegment reaches a first predetermined threshold.

In step 262, configuring the device to operate in the first mode ofoperation to measure affective response of the user to the firstsegment.

In step 263, receiving a second tag that indicates a second durationduring which to measure affective response to a second segment ofcontent by operating the device in the first mode of operation.

In step 264, receiving a second indication, derived from second eyetracking data of the user acquired in temporal vicinity of the secondduration, indicating that a gaze-based attention level to the secondsegment does not reach a second predetermined threshold.

And in step 265, configuring the device to operate in a second mode ofoperation to measure affective response of the user to the secondsegment.

In one embodiment, power consumption of the device when operating in thefirst mode of operation is significantly higher than the powerconsumption of the device when operating in the second mode ofoperation. Thus, operating the device in the second mode of operation,during at least part of the time tags indicated it should be operated inthe first mode of operation, may result is reducing the power consumedby the device.

In one embodiment, the first tag includes a representation of anexpected emotional response to being exposed to the first segment.Optionally, the method described in FIG. 2 includes an additional stepof forwarding, to a module that trains an emotional response model, thefirst tag and measurements of the affective response of the user tobeing exposed to the first segment. Optionally, the module that trainsan emotional response model generates a training sample formmeasurements of the affective response of the user to being exposed tothe first segment, and corresponding label is generated from the firsttag.

In one embodiment, the method described in FIG. 2 includes an additionalstep of generating the first segment and the first tag by an interactivecomputer game having an element whose actions in the game are at leastpartially controlled by the user. Optionally, the first tag correspondsto a certain event occurring in the first segment. For example, thefirst tag indicates to measure the user while the user is exposed to anexciting action sequence that takes place in the first segment.

Tags that correspond to segments of content, such as the one illustratedin FIG. 3, may have various properties and characteristics.

In one embodiment, a tag, such as the tags 292 a and/or 292 b, may beindicative of a condition and/or event that should occur in order formeasurements of affective response of a user to be taken with a device,such as the device 252. For example, a tag may be indicated that if auser selects a certain option, which leads to the user's exposure to aspecific segment of content, the user's affective response should bemeasured during a specified duration after the start of thepresentation.

In one embodiment, tags, such as the tags 292 a and/or 292 b, aregenerated substantially independently of behavior of the user 114 whenthe 114 user is exposed to segments of content corresponding to thetags. For example, the tags may be generated prior to the user'sexposure to the segments and provided as meta information correspondingto the segments (e.g., the tags may indicate to which scenes in a moviethe user's response should be measures, and they are determined beforethe movie is played to the user). In another example, a computer gamemay provide tags for certain portions of the game in which it isanticipated that users are likely to have noticeable affectiveresponses.

In one embodiment, a tag may contain information pertaining to a mode ofoperation in which a device, such as the device 252, may operate. Forexample, the tags 292 a and/or 292 b may identify the first mode ofoperation 287 a and/or a second mode of operation 287 b (e.g., bystating that the device should operate at a “normal” mode of operationor and “extensive measurement” mode of operation). In another example,the tags 292 a and/or 292 b may provide certain parameters pertaining tothe first mode of operation 287 a (e.g., desired sampling rate,resolution, and/or voltage to operate the device at). Alternatively oradditionally, tags may imply what the first mode of operation 287 ashould be. For example, by not providing instructions and/or detailpertaining to the first mode of operation, it may be implied that thefirst mode of operation 287 a should be a default mode of operation.

In one embodiment, tags may be received and/or sent individually, i.e.,they may be sent by a sender and/or received by a recipient one at atime. In one example, the tags 292 a and 292 b are sent one at a timeand/or are received by the first interface 254 one at a time.Additionally, in some embodiments, tags may be received sporadically,without the recipient of the tags having prior knowledge regarding wheneach tag is to be sent and/or how many tags to expect.

In another embodiment, tags, such as the tags 292 a and/or 292 b, may besent and/or received in batches, i.e., more than one tag is sent and/orreceived at a time (e.g., tags are sent in files or messages thatcontain multiple tags). For example, the first interface 254 may receivethe first and second tags together (e.g., they may be sent in the samefile). Optionally, the batches comprising multiple tags may be sentand/or received sporadically. Optionally, a recipient and/or sender ofbatches of tags may not have prior information of when batches are to besent and/or received, and/or how many tags and/or batches to expect.

In one embodiment, a tag, such as the tags 292 a and/or 292 b, mayinclude a representation of an expected emotional response of a user tothe segment of content. For example, the tag may describe a segment aslikely to excite, frighten, and/or bore a user.

In some embodiments, a device, such as the device 252, is used tomeasure affective response of the user 114. Optionally, the deviceincludes a sensor and/or is a sensor. Optionally, the sensor may be aphysiological sensor (e.g., a sensor that measures heart rate, galvanicskin response, and/or brainwave activity), and/or a sensor that measuresthe user's behavior (e.g., a camera, and/or a motion detector).Optionally, the device may include additional components to the sensor,such as a memory, a processor, a battery, a transmitter, and/or areceiver. Optionally, the device may be coupled to a user. Herein aphrase like “a device coupled to a user” refers to a device that isattached to the user (e.g., on clothing, a bracelet, headgear), incontact with the user's body (e.g., a sticker on a user's skin), and/orimplanted in the user's body (e.g., an implanted heart-rate sensor orimplanted EEG electrodes).

In one embodiment, the first and second eye tracking data are acquiredutilizing an eye tracker, such as the eye tracker 251, which includesimage capturing device. Optionally, the device 252 may be a batterypowered sensor that measures a physiological signal and is coupled tothe user. Optionally, the first and second eye tracking data arecollected substantially independently of whether the user is exposed tosegments of content identified by tags, and the device 252 comprises abattery powered sensor that is mainly used to measure affective responseof the user while exposed to segments identified by tags.

In one embodiment, the eye tracker 251 operates substantiallyindependently of whether the user 114 is exposed to segments of contentidentified by tags. Optionally, the device 252 is a battery poweredsensor that is mainly used to measure affective response of the userwhile exposed to segments identified by tags. In one example, by“substantially independently of whether the user is exposed . . . ” itis meant that the normal behavior, unless specifically overridden by theuser or software, is for the eye tracker 251 to operate without respectto tags. In another example, “substantially independently of whether theuser is exposed . . . ” it is meant that the eye tracker 251 has acontroller that may not necessarily take into consideration instructionsor indications that appear in tags.

In some embodiments, a battery refers to an object that can store energyand provide it in the form of electrical energy. In one example, abattery includes one or more electrochemical cells that convert storedchemical energy into electrical energy. In another example, a batteryincludes a capacitor that can store electrical energy.

Herein, a device is considered to be a battery powered device (alsoreferred to as a “battery operated device”), when a battery is able tosupply the device's power consumption while the device operates in aregular mode (i.e., not a sleep mode). Furthermore, the battery needs tobe able to meet the device's power demands for at least one second,without the battery or the device receiving power from an external powersource at that time (though at other times the device may receive powerfrom an external source). For example, the main battery of a mobilephone or a Bluetooth earpiece is considered a battery for the purpose ofthe present embodiments, but a current stabilizer or a capacitor, whichmay be able to operate a circuit for just a tiny fraction of a second,is not considered as a battery.

In one embodiment, a segment of content may include visual content, suchas images and/or video. A content delivery module may be used to exposethe segments to the user 114. Examples of content delivery modules viawhich the user may be exposed to visual content include a screenembedded in a mobile device such as a tablet or a smart phone, anon-mobile display such as a wall-mounted television, or a head-mounteddisplay such as virtual reality or augmented reality display glasses.

In one example, a device used to measure the user 114, such as thedevice 252, includes at least one component that is not shared with thecontent delivery module. The device in this example may be a braceletworn by the user 114 which can measure the user's heart rate, while thecontent delivery module is a screen embedded in the user's phone (whichis a different device than the bracelet).

In another example, a device used to measure the user 114, such as thedevice 252, may include at least one component that is shared with thecontent delivery module. In this example, a head-mounted device mayinclude both an EEG sensor and a display serving as a content deliverymodule, which can display augmented-reality, virtual reality, and/orvideo content, to the user.

In yet another example, a device used to measure the user 114, such asthe device 252, may be a battery powered sensor, and the contentdelivery module and the battery powered sensor are realized in the samedevice. For example, the user 114 may be wearing a computing devicenearby or on his wrist, such as a phone. The wrist-worn computing devicein this case may include both a sensor for measuring GSR and a flexibletouch screen display that serves as the content delivery module.

In one embodiment, at least one element of the controller 258 isimplemented as a dedicated hardware element. For example, the controller258 may be a component coupled to the eye tracker 251 and/or the device252. Alternatively or additionally, at least some of the functionalityof the controller 258 is implemented in software running on a generalpurpose processor (for example, the processor 256 may be a generalpurpose processor). In some cases, the controller 258 may run, at leastin part, on a device belonging to the user 114, such as a mobile phone.In some cases, the controller 258 may run, at least in part, on a systemremote of the user, such as on a cloud-based server.

In one embodiment, the controller 258 is realized as hardware that isembedded in the housing of the device 252. For example, a controller fora mobile EEG measuring headset may be embedded in the headset itself,and communicate data wirelessly. Additionally, at least some of thefunctionality of the controller 258 may be realized as software thatruns on a processor that belongs to the device 252 and/or the eyetracker 251. In one example, the controller for a device that is amobile EEG measuring headset may run on a processor that belongs to theheadset and may be used to process EEG measurements.

In one embodiment, the system 250 includes a measurement EmotionalResponse Predictor (measurement ERP) configured to predict the user'semotional response from data comprising measurements of affectiveresponse of the user 114 taken with the device 252. Optionally, themeasurements of the device 252 are processed prior to being sent to themeasurement ERP for prediction. For example, the measurements mayundergo filtering and/or feature extraction.

FIG. 4 illustrates one embodiment that includes a model trainer 290 fortraining a model for a measurement Emotional Response Predictor(measurement ERP). The model trainer 290 receives as input themeasurements taken by a device that measures the user 114, such as themeasurements 252 a taken by the device 252. The received measurementsare taken in temporal vicinity of the first duration indicated by thetag 292 a. The measurements include affective response of the user 114to the first segment of content corresponding to the tag 292 a.Optionally, the measurements may be processed prior to being sent to themodule to be used for training; for instance, the measurements undergofiltering and/or feature extraction in order to produce feature valuesthat are included in one or more training samples utilized by themodule.

In one embodiment, a tag may include a value indicative of an expectedemotional response to a segment of content to which the user 114 wasexposed essentially during a duration indicated by the tag. Thus, themodel trainer 290 may be used to create training samples that includemeasurement values along with the corresponding emotional response auser is likely to experience. The training samples can then be utilizedby a machine learning-based algorithm that trains an emotional responsemodel 291.

In one embodiment, a training sample that includes values derived frommeasurements of a user, along with the corresponding emotional response,is generated from the tag 292 a and measurements received by the modeltrainer 290 (e.g., the measurements 252 a). The training sample isprovided to an algorithm that trains the emotional response model 291that may be used for predicting emotional response from measurements ofaffective response. For example, algorithm may train a model utilized bya predictor such as a neural network, Naive Bayes classifier, and/ordecision trees. Optionally, the trained model is utilized by ameasurement ERP to predict emotional response from certain measurementsof affective response. Optionally, the certain measurements of affectiveresponse utilized by the measurement ERP to make predictions ofemotional response are measurements taken by a device that measures theuser, such as the device 252.

FIG. 5 illustrates an interactive computer game 294 that generates thesegments (289 a to 289 d) and the tags (283 a to 283 c). The interactivecomputer game has an element whose actions in the game are at leastpartially controlled by the user 114. Being able to measure theaffective response of the user 114 to segments of content (289 a to 289d) generated by the computer game can be beneficial since it can helpguide the system towards alterations that can improve the user's playingexperience. For example, if the user's affective response indicates thatthe user is losing interest and is not excited by playing at a certainlevel of difficulty, the game can increase the difficulty and addsurprising effects and/or plot twists in order to make the game moreexciting for the user.

In one embodiment, at least some of the tags 283 a to 283 c aregenerated by the interactive computer game 294 are generated in order togauge affective response of the user 114 to events occurring in a gamewhich the user 114 is playing. The computer game 294 may generate a tagthat is indicative of a duration during which a device is to measureaffective response of the user, and/or indicate a mode of operation forthe device to operate in. For example, tag 283 a may indicate that thedevice 252 is to be operated at a first mode of operation 287 a in orderto measure the user's affective response to the segment 289 a.

In one embodiment, measurements of the user, taken by a device, such asthe device 252, during a duration indicated by a tag may be utilized bythe interactive computer game in order to determine emotional responseof the user 114 (e.g., by providing the measurements to a measurementERP). The predicted emotional response may be used in order to determineif elements in the game need to be changed in order to improve the userexperience. For example, if in an action game, the measurements of theaffective response of the user 114 reveal that the user is becomingfrustrated, possibly due to not being able to complete a level, thecomputer game may reduce the difficulty of the level to allow the userto succeed. In another example, the user 114 controls a character in avirtual world that is part of a larger group of other characters thatmay be controlled by the computer game. In this game, some of thecharacters may be killed off, e.g., in a fight with monsters in thevirtual world. The computer game may generate tags that initiatemeasuring of the affective response of the user 114 when interactingwith different virtual characters in order to determine which charactersthe user 114 likes. Using this information, the computer game may selectspecific characters, such as ones the user does not in particularlylike, to be characters that fall prey to a rampaging dragon.

In one embodiment, the functionality of the controller 258 is includedin the interactive computer game 294. In this embodiment, the computergame sets a mode of operation for the device 252. For example, thecontroller 258 is implemented, at least in part, as software running ona processor belonging to a game console. In another example, thecontroller 258 is implemented, at least in part, as software running ona server running a game such as an online multiplayer game.

FIG. 6 illustrates a controller 288 that utilizes measurements of asensor 281 to reduce power consumption of a device 282 operative tomeasure affective response. The controller 288 includes at least a firstinterface 284, a second interface 285, and a processor 286.

The first interface 284 receives tags, such as the tags illustrated inFIG. 4, which correspond to segments of content to which the user 114 isexposed. Optionally, the first interface 284 is implemented, at least inpart, in software. Optionally, the software runs on the processor 286.In one embodiment, the first interface 284 is configured to receive atleast a first tag 292 a and a second tag 292 b: The first tag 292 aindicates a first duration during which to measure affective response toa first segment of content by operating the device 282 in a first modeof operation. The second tag 292 b indicates a second duration duringwhich to measure affective response to a second segment of content byoperating the device 282 in the first mode of operation.

The second interface 285 receives measurements of the user 114 taken bythe sensor 281 in temporal vicinity of durations indicated by the tags.Optionally, the second interface 285 is implemented, at least in part,in software. Optionally, the software runs on the processor 286. In oneembodiment, the second interface 285 is configured to receive at leastfirst and second measurements: The first measurement is a measurement ofthe user 114 taken by the sensor 281 in temporal vicinity of the firstduration indicated by the first tag 292 a. The second measurement is ameasurement of the user 114 taken by the sensor 281 in temporal vicinityof the second duration indicated by the second tag 292 b.

In one embodiment, measurements of the user 114 taken by the sensor 281in temporal vicinity of the durations are processed prior to beingevaluated by the processor 286. For example, the measurements mayundergo filtration, normalization, and/or feature extraction.Optionally, at least some of the processing of the measurements isperformed by the processor 286. Alternatively or additionally, at leastsome of the processing is performed by the interface 285 and/or thesensor 281.

The processor 286 selects a mode of operation for the device 282, basedon the evaluation of measurements of the user 114 taken by the sensor281 in temporal vicinity of durations indicated by tags. The processor286 may compare the measurements and/or data derived from themeasurements to a predetermined threshold in order to determine whetherthe predetermined threshold is reached. Thus, given a duration indicatedby a tag, the processor 286 may assign an appropriate mode of operationbased on whether or not the measurements of the sensor 281, whichcorrespond to the duration, reach the predetermined threshold. In oneembodiment, the processor 286 is configured to receive first and secondmeasurements taken by the sensor 281. Optionally, the processor 286 maydetermine that the first measurement reaches a first predeterminedthreshold, and accordingly selects a first mode of operation 287 a forthe device 282 to operate in order to measure affective response of theuser 114 to the first segment during the first duration. Additionally oralternatively, the processor 286 may determine that the secondmeasurement does not reach a second predetermined threshold, andaccordingly selects a second mode of operation 287 b for the device 282to operate in order to measure affective response of the user 114 to thesecond segment during the second duration.

Herein, by stating that a value reaches a predetermined threshold, it ismeant that the value equals or exceeds the predetermined threshold. Inaddition, a predetermined threshold to which a value a measurement ofaffective response is compared, refers to a value of which there isprior knowledge. For example, the threshold value itself is known and/orcomputed prior to when the comparison is made. Additionally oralternatively, a predetermined threshold may utilize a threshold valuethat is computed according to logic (such as function) that is knownprior to when the comparison is made.

In one embodiment, the first predetermined threshold and the secondpredetermined threshold may represent different thresholds. For example,reaching the first predetermined threshold is based on achievingdifferent value than is required to pass the second predeterminedthreshold. Thus, there may be cases in which a certain value may reachthe first predetermined threshold, but not reach the secondpredetermined threshold. Alternatively, in another embodiment, the firstpredetermined threshold and the second predetermined threshold mayrepresent essentially the same predetermined threshold. Thus, in thisembodiment, a value that reaches the first predetermined threshold alsoreaches the second predetermined. Similarly, in this embodiment, a valuethat does not reach the first predetermined threshold also does notreach the second predetermined threshold.

In one embodiment, a predetermined threshold is a value of aphysiological signal. For example, the sensor may measure heart rate,and the predetermined threshold is a value of a specific heart rate. Ifthe sensor measures a heart rate that exceeds the value of thepredetermined threshold, the threshold is considered to be reached. Inanother example, the predetermined threshold is a value representing aspecific level of electrical conductance of skin, as measured by a GSRsensor (Galvanic Skin Response). In yet another example, thepredetermined threshold is a level of movement, and the sensor is anaccelerometer.

In another embodiment, a predetermined threshold is a value that iscomputed in temporal vicinity to when measurements of the user 114 aretaken by a sensor. For example, the predetermined threshold may be aheart rate that is a function of a baseline value that is a runningaverage heart rate of the preceding 10 minutes. In this example, thepredetermined threshold may me set to represent a significant increaseover the baseline; for instance, the predetermined threshold is set tobe 115% of the baseline value. In another example, the predeterminedthreshold may be a level of skin conductivity that is proportional tothe maximum value previously detected for the user 114.

In yet another embodiment, a predetermined threshold is derived from abehavioral cue. In one example, the predetermined threshold mayrepresent a minimum duration in which the user does not make substantialmovements (which could be detected by sensors such as cameras or EMG).In another example, the predetermined threshold is based on detecting aspecific facial expression such as a smile or a cringe (e.g., from beingexposed to a frightening scene). In this example, if the requiredexpression is identified, e.g., by analyzing images captured with acamera, the threshold may be considered to be reached.

In still another embodiment, a predetermined threshold includes a valuerepresenting an emotional response that can be computed from datacomprising measurements of a sensor, such as the sensor 281. Optionally,the emotional response is predicted from measurements of the sensorutilizing a measurement ERP (Emotional Response Predictor). In oneexample, the predetermined threshold may involve a specific level ofarousal, which is computed from measurements of the sensor which in thisexample may be a GSR sensor. In another example, the predeterminedthreshold involves a minimal level of interest the user should displayin order for the threshold to be reached. In this example, the interestmay be predicted from images captured by the sensor (e.g., the sensormay be a camera).

In one embodiment, computing a predetermined threshold is done utilizingadditional information to measurements received from a sensor, such asthe sensor 281. In one example, the predetermined threshold may becomputed using external information that pertains to the context orsituation the user is in, e.g., information indicating what activity isthe user partaking in and/or what type of media the content is. Inanother example, a predetermined threshold involving a heart rate maychange depending on the level of the user's activity at the time (e.g.,walking in the park or sitting on the couch). In yet another example,computing the predetermined threshold may utilize historical informationsuch as medical records, in order to determine the predeterminedthreshold. For example, predetermined thresholds involving brainwavemeasurements may consult medical records of a user in order to determinewhether the threshold needs to be adjusted according to certain mentalconditions or personality traits of the user.

In one embodiment, the processor 286 selects a mode of operation, fromtwo modes of operation: a first mode of operation that is a standby modeof operation, and a second mode of operation that is a normal mode ofoperation. In this example, as is the case with many devices, when thedevice 282 operates in the standby mode, it consumes significantly lesspower compared to when it operates in the normal mode of operation.

In another embodiment, selecting a mode of operation may involveselecting different parameter settings for the operation of a device,such as the device 282.

In one example, a predetermined threshold that is used may indicatewhether a user is paying attention or not (e.g., as detected by a sensorthat measures the rate of the user's movements while sitting down). Theprocessor 286 may select, according to whether the predeterminedthreshold was reached, parameters that define the mode of operation,such as a resolution of images captured by the device which may be acamera. If the predetermined threshold is reached, the processor mayselect a first mode of operation in which the camera captureshigh-resolution images (e.g., 1920×1080 pixels). In this case, usinghigh resolution images may enable better analysis of the user'saffective response, since it can enable better detection of gesturesand/or facial expressions. However, If the predetermined threshold isnot reached (indicating that the user is not paying sufficientattention), the processor may select a second mode of operation in whicha camera captures low-resolution images (e.g., 320×240 pixels), whichenables only basic image analysis. In some cases, different resolutionslead to different power consumptions for acquiring images. For instance,the lower resolution of the second mode of operation can cause a camerato utilize significantly less CMOS gates, compared to the number of CMOSgates it needs to utilize in the first mode of operation. Thus, in thisexample, the power consumption of the device operating in the first modeof operation is significantly higher than the power consumption of thedevice operating in the second mode of operation.

In another example, the predetermined threshold may involve reaching acertain level of excitement, for example, as determined by an arousallevel measured by a GSR sensor. In this example, the user 114 ispresented with content that can frighten and/or excite the user. Thedevice 282 may be a battery operated headset that measures brainwaveswith EEG. If the threshold is reached, it means that the content beingviewed by the user 114 at the time causes the user to be excited; thus,the processor 286 may select a first mode of operation in which theuser's brainwaves are measured extensively (e.g., by measuring multiplebands of frequencies). Measuring the user with the EEG may helpdetermine to what extent the user was really frightened while watchingthe content, and maybe better characterize the emotions the user felt.However, if the threshold is not reached, it is not likely that the user114 is frightened or excited by the content, and thus the processor maychoose a second mode of operation in which the user is not measured atall, or measured rudimentary (e.g., by monitoring less bands offrequencies).

In one embodiment, the sensor 281 comprises an image capturing deviceand the device 282 is a battery powered sensor that measures aphysiological signal and is coupled to the user (e.g., GSR or EEG).Optionally, the measurements of the camera are utilized in order todetermine whether the user is paying attention to content, and the modeof operation for the device 282 is selected accordingly.

In another embodiment, the sensor 281 is coupled to the user 114 andmeasures a physiological signal.

In yet another, the device 282 may be a battery powered sensor that iscoupled to the user, and measures a physiological signal. For example,the device 282 may be a GSR sensor embedded in a bracelet or an EEGsensor implanted in the scalp or embedded in headgear.

In one embodiment, the sensor 281 operates substantially independentlyof whether the user 114 is exposed to segments of content identified bytags; and the device 282 is a battery powered sensor that is mainly usedto measure affective response of the user while exposed to segmentsidentified by tags. In one example, the sensor 281 may be a movementsensor (e.g., an accelerometer in a mobile phone that workssubstantially all the time), and the device 282 may be a batteryoperated EEG sensor embedded in headgear. In another example, the sensor281 may be a camera embedded in a television, such as a camera thatenables gesture control of the television and thus operatessubstantially whenever the television is on; the device 282 in thisexample may be bracelet with embedded sensors that monitor heart rateand/or GSR. In one example, by “substantially independently of whetherthe user is exposed . . . ” it is meant that the normal behavior, unlessspecifically overridden by the user or software, is for the sensor 281to operate without respect to tags. In another example, “substantiallyindependently of whether the user is exposed . . . ” it is meant thatthe sensor 281 has a controller that may not necessarily take intoconsideration instructions or indications that appear in tags.

In one embodiment, a segment of content may include visual content, suchas images and/or video. A content delivery module may be used to exposethe segments to the user 114. Optionally, the device 282 has at leastone component that is not shared with the content delivery module.Additionally or alternatively, the device 282 may include at least onecomponent that is shared with the content delivery module. Additionallyor alternatively, the device 282 may be a battery powered sensor, andthe content delivery module and the battery powered sensor are realizedin the same device.

In one embodiment, at least one element of the controller 288 isimplemented as a dedicated hardware element. For example, the controller288 may be a component coupled to the sensor 281 and/or the device 282.Alternatively or additionally, at least some of the functionality of thecontroller 288 is implemented in software running on a general purposeprocessor (for example, the processor 286 may be a general purposeprocessor). In some cases, the controller 288 may run, at least in part,on a device belonging to the user 114, such as a mobile phone. In somecases, the controller 288 may run, at least in part, on a system remoteof the user, such as on a cloud-based server.

In one embodiment, the controller 288 is realized as hardware that isembedded in the housing of the sensor 281 and/or is part of the device282. For example, the controller for a mobile EEG measuring headset maybe embedded in the headset itself, and communicate data wirelessly.Additionally, at least some of the functionality of the controller 288may be realized as software that runs on a processor that belongs to thesensor 281 and/or the device 282. For example, the controller for amobile EEG measuring headset may run on a processor that belongs to theheadset and is used to process EEG measurements.

In one embodiment, the system 280 includes a measurement EmotionalResponse Predictor (measurement ERP) configured to predict the user'semotional response from data comprising measurements of affectiveresponse of the user 114 taken with the device 282. Optionally, themeasurements taken by the device are processed prior to being sent tothe measurement ERP for prediction. For example, the measurements mayundergo filtering and/or feature extraction. Additionally, themeasurement ERP may receive and utilize measurements of affectiveresponse of the user 114 taken with the sensor 281 during at least someof the durations in which the measurements of the user 114 were takenwith the device 282.

In one embodiment, the system 280 includes a model trainer 290, asillustrated in FIG. Optionally, the model trainer 290 may receive asinput measurements taken by the device 282, in temporal vicinity of thefirst duration indicated by the tag 292 a. The measurements may includeaffective response of the user 114 to the first segment of contentcorresponding to the tag 292 a.

In one embodiment, the computer game 294, which is illustrated in FIG.5, generates the tags 292 a and 292 b received by the first interface284. Measurements taken with the device 282 during the first duration,indicated by the tag 292 a, may then be utilized by the interactivecomputer game in order to determine emotional response of the user 114(e.g., by providing the measurements to a measurement ERP). Thepredicted emotional response may be used in order to determine ifelements in the game need to be changed in order to improve the userexperience.

In one embodiment, the functionality of the controller 288 is includedin the interactive computer game 294. In this embodiment, the computergame sets a mode of operation for the device 282. For example, thecontroller 288 is implemented, at least in part, as software running ona processor belonging to a game console. In another example, thecontroller 288 is implemented, at least in part, as software running ona server running a game such as an online multiplayer game.

FIG. 7 illustrates one embodiment of a method for utilizing a sensor toreduce power consumption of a device used to measure affective response.The method includes the following steps:

In step 300, receiving a first tag that indicates a first durationduring which to measure affective response to a first segment of contentby operating the device in a first mode of operation.

In step 301, receiving a first indication that a first measurement ofthe user, taken by the sensor in temporal vicinity of the firstduration, reaches a first predetermined threshold. Optionally, the firstmeasurement is taken during a period that ends after the end of theexposure of the user to the first segment. Alternatively oradditionally, the first measurement may be taken during a period thatstarts before the start of the exposure of the user to the firstsegment.

In step 302, configuring the device to operate in the first mode ofoperation while measuring affective response of the user to the firstsegment.

In step 303, receiving a second tag that indicates a second durationduring which to measure affective response to a second segment ofcontent by operating the device in the first mode of operation.

In step 304, receiving a second indication that a second measurement ofthe user, taken by the sensor in temporal vicinity of the secondduration, does not reach a second predetermined threshold. Optionally,the second measurement is taken during a period that ends after the endof the exposure of the user to the second segment. Alternatively oradditionally, the second measurement may be taken during a period thatstarts before the start of the exposure of the user to the secondsegment.

An in step 305, configuring the device to operate in a second mode ofoperation while measuring affective response of the user to the secondsegment.

In one embodiment, power consumption of the device when operating in thefirst mode of operation is significantly higher than the powerconsumption of the device when operating in the second mode ofoperation. Thus, operating the device in the second mode of operation,during at least part of the time tags indicated it should be operated inthe first mode of operation, may result is reducing power consumed bythe device.

In one example, the first predetermined threshold may involve a specificmeasurement value such as a heart rate. In this example, the firstindication indicates that the heart rate of the user 114 measured by thesensor in temporal vicinity of the first duration, reaches thepredetermined threshold's value. Additionally, the second predeterminedthreshold may also involve a specific measurement value such as a heartrate (this value may optionally be different than the value involved inthe first predetermined threshold). In this example, the secondindication may indicate that the heart rate of the user 114 measured bythe sensor in temporal vicinity of the second duration, does not reachthe second predetermined threshold's value.

In another example, the first and second predetermined thresholds mayinvolve, possible different, emotional responses a user should feel.Optionally, the prediction of emotional response is performed by theprocessor utilizing a measurement ERP (Emotional Response Predictor). Inthis example, measurement of the affective response of the user takenwith the sensor is provided to the processor in order to predict theuser's likely emotional response, and generate indications accordingly.Optionally, the indications are used internally by the processor, forinstance, by passing values and/or messages between different proceduresand/or regions in memory that are involved in running the controller. Inthis example, the first indication indicates that the user likely feelsthe required emotional response according to the first predeterminedthreshold, so the first measurement taken in temporal vicinity of thefirst duration reaches the first predetermined threshold. In contrast,the second indication indicates that the user does not likely feel therequired emotional response according to the second predeterminedthreshold, so the second measurement taken in temporal vicinity of thesecond duration does not reach the second predetermined threshold.

In one embodiment, the first tag includes a representation of anexpected emotional response to being exposed to the first segment.Optionally, the method described in FIG. 7 includes an additional stepof forwarding, to a module that trains an emotional response model, thefirst tag and measurements of the affective response of the user tobeing exposed to the first segment. Optionally, the module that trainsan emotional response model generates a training sample formmeasurements of the affective response of the user to being exposed tothe first segment and corresponding label is generated from the firsttag.

In one embodiment, the method described in FIG. 7 includes an additionalstep of generating the first segment and the first tag by an interactivecomputer game having an element whose actions in the game are at leastpartially controlled by the user. Optionally, the first tag correspondsto a certain event occurring in the first segment. For example, thefirst tag indicates to measure the user while the user is exposed to anexciting action sequence that takes place in the first segment.

In one embodiment, a controller configured to utilize measurements of asensor to reduce power consumption of a device operative to measureaffective response, comprising: a first interface configured to receivea first tag that indicates a first duration during which to measureaffective response to a first segment of content by operating the devicein a first mode of operation; a second interface configured to receive afirst measurement of a user, taken by the sensor in temporal vicinity ofthe first duration; and a processor configured to: determine that thefirst measurement reaches a first predetermined threshold, and to selectthe first mode of operation for operating the device, during the firstduration, to measure affective response of the user to the firstsegment; the first interface is further configured to receive a secondtag that indicates a second duration during which to measure affectiveresponse to a second segment of content by operating the device in thefirst mode of operation; the second interface is further configured toreceive a second measurement of the user taken by the sensor in temporalvicinity of the second duration; the processor is further configured todetermine that the second measurement does not reach a secondpredetermined threshold, and to select a second mode of operation foroperating the device during the second duration while measuringaffective response of the user to the second segment; wherein powerconsumption of the device when operating in the first mode of operationis significantly higher than power consumption of the device whenoperating in the second mode of operation. Optionally, the first tagfurther comprises a representation of expected emotional response to thefirst segment. Optionally, the controller further comprises a trainingmodule configured to receive the first tag and measurements taken by thedevice during the first duration, and to train an emotional responsemodel utilizing the first tag and the measurements. Optionally, thefirst tag depends on information conveyed by the first segment, and doesnot depend mainly on behavior of the user while being exposed to thefirst segment. Optionally, the sensor comprises an image capturingdevice and the device is a battery powered sensor that measures aphysiological signal and is coupled to the user. Optionally, the deviceis a battery powered sensor that is coupled to the user and measures aphysiological signal. Optionally, the sensor is coupled to the user andmeasures a physiological signal of the user. Optionally, the sensoroperates substantially independently of whether the user is exposed tosegments of content identified by tags; and the device is a batterypowered sensor that is mainly used to measure affective response of theuser while the user is exposed to segments identified by tags.Optionally, the first and second segments and the first and second tagsare generated by an interactive computer game having an element whoseactions in the game are at least partially controlled by the user.Optionally, the power consumption of the device when operating in thesecond mode of operation is essentially zero.

In one embodiment, a method for utilizing a sensor to reduce powerconsumption of a device used to measure affective response, comprising:receiving a first tag that indicates a first duration during which tomeasure affective response to a first segment of content by operatingthe device in a first mode of operation; receiving a first indicationthat a first measurement of a user, taken by the sensor in temporalvicinity of the first duration, reaches a first predetermined threshold;configuring the device to operate in the first mode of operation tomeasure affective response of the user to the first segment; receiving asecond tag that indicates a second duration during which to measureaffective response to a second segment of content by operating thedevice in the first mode of operation; receiving a second indicationthat a second measurement of the user, taken by the sensor in temporalvicinity of the second duration, does not reach a second predeterminedthreshold; and configuring the device to operate in a second mode ofoperation to measure affective response of the user to the secondsegment; wherein power consumption of the device when operating in thefirst mode of operation is significantly higher than the powerconsumption of the device when operating in the second mode ofoperation. Optionally, the first tag further comprises a representationof an expected emotional response to being exposed to the first segment.Optionally, the method further comprises forwarding, to a module thattrains an emotional response model, the first tag and measurements ofthe affective response of the user to being exposed to the firstsegment. Optionally, the first tag depends on information conveyed bythe first segment, and does not depend mainly on behavior of the userwhile being exposed to the first segment. Optionally, the sensorcomprises an image capturing device and the device is a battery poweredsensor that measures a physiological signal and coupled to the user.Optionally, the device is a battery powered sensor that is coupled tothe user and measures a physiological signal. Optionally, the sensor iscoupled to the user and measures a physiological signal of the user.Optionally, the sensor operates substantially independently of whetherthe user is exposed to segments of content identified by tags; and thedevice is a battery powered sensor that is mainly used to measureaffective response of the user while exposed to segments identified bytags. Optionally, the method further comprises generating the firstsegment and the first tag by an interactive computer game having anelement whose actions in the game are at least partially controlled bythe user; wherein the first tag corresponds to a certain event occurringin the first segment.

In one embodiment, a controller configured to utilize a sensor to reducepower consumption of a device operative to measure affective response,comprising: a first interface configured to receive a first tag thatindicates a first duration during which to measure affective response toa first segment of content by operating the device in a first mode ofoperation; wherein the device is battery powered and measures aphysiological signal; a second interface configured to receive a firstmeasurement of a user taken by the sensor in temporal vicinity of thefirst duration; and a processor configured to: determine that the firstmeasurement reaches a first predetermined threshold, and to select thefirst mode of operation for operating the device during the firstduration while measuring affective response of the user to the firstsegment; the first interface is further configured to receive a secondtag that indicates a second duration during which to measure affectiveresponse to a second segment of content by operating the device in thefirst mode of operation; the second interface is further configured toreceive a second measurement of the user taken by the sensor in temporalvicinity of the second duration; the processor is further configured todetermine that the second measurement does not reach a secondpredetermined threshold, and to select a second mode of operation foroperating the device during the second duration while measuringaffective response of the user to the second segment; wherein powerconsumption of the device when operating in the first mode of operationis significantly higher than power consumption of the device whenoperating in the second mode of operation.

FIG. 8 illustrates one embodiment of a system 350 that is configured toreduce power consumption of a device 358 by overriding instructions tomeasure indicated by tags. The system 350 includes at least amode-selector 352 and a threshold module 362.

The mode-selector 352 is configured to receive tags corresponding tosegments of content, such as the tags illustrated in FIG. 3. In oneembodiment, the mode-selector 352 is configured to receive the first tag292 a and the second tag 292 b corresponding to first and secondsegments of content, respectively. The first tag 292 a indicates tooperate the device 358 in a first mode of operation 357 a to measureaffective response to the first segment. The second tag 292 b indicatesto operate the device 358 in a first mode of operation 357 a to measureaffective response to the second segment.

The threshold module 362 is configured to receive, from a sensor 360,measurements of affective response of the user 114, taken in periodcorresponding to an exposure of the user to segments of content.Optionally, a measurement corresponding to a certain segment of contentis taken essentially before the end of exposure of the user to thesegment. Thus, the measurement may reflect a condition of the user 114during the time of exposure to a segment of content, such as a state ofaffective response of the user. Optionally, the threshold module 362 isalso configured to determine whether the measurement reaches apredetermined threshold.

In one embodiment, a measurement taken essentially before end ofexposure of the user to a segment of content means that the measurementis taken during a period spanning when the user is exposed to thesegment, and possibly before that. For example, the measurement may betaken over a period spanning from 10 seconds before exposure to thesegment starts and until half of the segment has been exposed to theuser. Optionally, a period in which the measurement is taken may alsoextend a bit after the end of exposure, for example, a measurement taken5 seconds after the end of exposure may still be considered as beingtaken essentially before end of exposure.

In another embodiment, a measurement of a user 114 taken by a sensor,such as the sensor 360, essentially before end of exposure of the userto a segment of content, may be taken during a period that starts, andpossibly ends, prior to the beginning of the exposure to the segment.Thus, for example, a measurement taken with the sensor 360 may beindicative of the state and/or expected state of the user 114 during theexposure to the segment of content.

Measurements of the user 114 taken by a sensor, such as the sensor 360,may be processed, in some embodiments, prior to being evaluated by thethreshold module 362. For example, the measurements may undergofiltration, normalization, and/or feature extraction. Optionally, atleast some of the processing of the measurements is performed by thethreshold module 362. Alternatively or additionally, at least some ofthe processing is performed by the sensor 360.

In one embodiment, the threshold module 362 is configured to receive afirst measurement 361 a and a second measurement 361 b of a user takenby the sensor 360 essentially before end of exposure of the user to thefirst and second segments, respectively. That is, the first measurement361 a is taken essentially before the end of exposure of the user to thefirst segment, and the second measurement 361 b is taken essentiallybefore the end of exposure of the user to the second segment.Optionally, the threshold module 362 may determine that the firstmeasurement 361 a reaches a first predetermined threshold, and mayoptionally indicate thereof to the mode-selector 352 (i.e., that thefirst measurement 361 a reaches the first predetermined threshold).Optionally, the threshold module 362 may determine that the secondmeasurement 361 b does not reach a second predetermined threshold, andmay optionally indicate to the mode-selector 352 thereof (i.e., that thesecond measurement 361 b does not reach the second predeterminedthreshold).

In one example, a predetermined threshold is a value of a physiologicalsignal. In another example, the predetermined threshold is a value thatis computed in temporal vicinity to when measurements of the user 114are taken by a sensor. In yet another example, a predetermined thresholdis derived from a behavioral cue. In still another example, apredetermined threshold includes a value representing an emotionalresponse that can be computed from data comprising measurements of asensor, such as the sensor 360. Optionally, computing a predeterminedthreshold is done utilizing additional information to the measurementsreceived from a sensor, such as the sensor 360.

The threshold module 362 may indicate to the mode-selector 352 whether ameasurement taken by the sensor 360 reaches a predetermined threshold.There may be different ways in which the threshold module 362 may makesuch indications. In one example, the threshold module 362 signals(e.g., via a message communicated to the mode-selector 352) a value thatdescribes if the measurement reached the threshold, and/or the actualmeasurement value and/or the difference between the measurement valueand the threshold. In another example, the threshold module 362 sends asignal to the mode-selector 352 when the measurement reaches thethreshold, and if no signal is sent, the mode-selector 352 operatesunder the assumption that the measurement did not reach the threshold.Alternatively, the threshold module 362 may send a signal to themode-selector 352 when the measurement does not reach the threshold, andif no signal is sent, the mode-selector 352 operates under theassumption that the measurement reached the threshold.

The mode-selector 352 is configured to select a mode of operation forthe device 358 to operate in when measuring affective response of theuser 114. Optionally, the selection of a mode of operation may be doneaccording to an indication in a received tag, such as the tags 292 aand/or 292 b. Optionally, the mode-selector 352 may make a selection ofthe mode of operation also according to other information such as anindication from the threshold module 362 regarding whether or not ameasurement value of the sensor 360 reaches a predetermined threshold.Optionally, a mode of operation for the device 358 is chosen from a setthat includes the first mode of operation 357 a and a second mode ofoperation 357 b.

In one embodiment, the mode-selector 352 selects the first mode ofoperation 357 a, as indicated by the first tag 292 a, for measuring theaffective response of the user to the first segment of content. Themode-selector makes this selection because the threshold moduleindicates that the measurement 361 a reaches a first predeterminedthreshold. However, the mode-selector 352 overrides the first mode ofoperation 357 a, indicated by the second tag 292 b, based on indicationof the threshold module 362 that the second measurement 361 b does notreach a second predetermined threshold. The power consumption of thedevice 358 when operating in the second mode of operation 357 b issignificantly lower than the power consumption of the device 358 whenoperating in the first mode of operation 357 a. Thus, by switching modesof operation of the device 358, the mode-selector 352 is able to helpreduce the power consumption of the device 358.

In one example, the mode-selector 352 may override a tentative choice ofthe first mode of operation 357 a (as indicated by a tag), by replacingthe selected first mode of operation 357 a with the second mode ofoperation 357 b. For instance, this may be done by changing parametersoriginally set by the mode-selector 352 according to informationreceived from the threshold module 362. In another example, overridingmay be achieved by the threshold module 362 running before themode-selector 352 receives a tag indicating the desired mode ofoperation. If the threshold module 362 determines that a measurement ofthe user taken with the sensor 360 does not reach a predeterminedthreshold, it may prevent the tag from reaching the mode-selector 352,thus casing the mode-selector 352 not to select the first mode ofoperation 357 a for the device 358. In this example, the defaultbehavior of the device 358 may be to operate in the second mode ofoperation 357 b; thus blocking the tag from reaching the mode-selector352 and/or blocking the mode-selector 352 from acting according to thetag is equivalent to selecting the second mode of operation for thedevice 358.

In some embodiments, the power consumption of the device 358 whileoperating in the first mode of operation 357 a may be significantlyhigher than its power consumption while operating in the second mode ofoperation 357 b. Consequently, by selecting for certain durations, thesecond mode of operation 357 b for the device 358 to operate in, insteadof the intended first mode of operation 357 a, the total amount ofenergy consumed over time by the device 358 may be reduced.Additionally, the mode-selector 352 may reduce the power consumption ofother elements that participate in the processing of the affectiveresponse measurements taken by the device 358, such as a processor thatprocesses the data comprising the measurements. Optionally, the powerconsumption of the device 358 while operating at the second mode ofoperation 357 b is essentially zero. For example, by essentially zero itis meant that the power consumption is at most 5% of the powerconsumption of the device 358 when operating at the first mode ofoperation 357 a.

In one embodiment, the mode-selector 352 selects a mode of operation,from two modes of operation: a first mode of operation that is a standbymode of operation, and a second mode of operation that is a normal modeof operation. In this example, as is the case with many sensors and/ordevices, when the device 358 operates in the standby mode, it consumessignificantly less power compared to when it operates in the normal modeof operation.

In another embodiment, selecting a mode of operation, by themode-selector 352, may involve selecting different parameter settingsfor the operation of a device, such as setting resolution of imagescaptured by a camera, or brainwave frequencies that are to be monitoredusing EEG.

In one embodiment, the system 350 includes a measurement EmotionalResponse Predictor (measurement ERP) configured to predict the user'semotional response from data comprising measurements of affectiveresponse of the user 114 taken with the device 358.

In one embodiment, the system 350 includes a module for training a modelfor a measurement Emotional Response Predictor (measurement ERP), suchas the model trainer 290 illustrated in FIG. 4. The module receives asinput measurements of affective response of the user 114 to segment ofcontent corresponding to tags, which were taken by the device 358.Optionally, the measurements may be processed prior to being sent to themodule to be used for training; for instance, the measurements undergofiltering and/or feature extraction in order to produce feature valuesthat are included in one or more training samples utilized by themodule. The module may also receive tags as input. For example, the tag292 a may include values indicative of an expected emotional response tothe first segment of content to which the user 114 was exposed while themeasurements were taken. Thus, the module can create a training samplethat includes measurement values along with the corresponding emotionalresponse a user is likely to experience. The training sample can then beutilized by a machine learning-based algorithm that trains an emotionalresponse model.

In one embodiment, the sensor 360 comprises an image capturing deviceand the device 358 is a battery powered sensor that measures aphysiological signal and is attached to or implanted in the user (e.g.,GSR or EEG). Optionally, the image capturing device is directed at alocation in which the user is expected to be in order to look at adisplayed content. Images captured by the device 358 may be utilized inorder to determine whether the user is present and/or paying attentionto the content, and the mode of operation for the device 358 is selectedaccordingly.

In one embodiment, the device 358 is a battery powered sensor that isattached to or implanted in the user, and measures a physiologicalsignal (e.g., a bracelet with an embedded GSR sensor, or headgear withembedded EEG sensors). Additionally, the sensor 360 may be a sensor thatis attached to or implanted in the user, and measures a physiologicalsignal (e.g., a bracelet with an embedded GSR sensor, or headgear withembedded EEG sensors).

In one embodiment, the sensor 360 operates substantially independentlyof whether the user 114 is exposed to segments of content identified bytags. Optionally, the device 358 is a battery powered sensor that ismainly used to measure affective response of the user while exposed tosegments identified by tags. For example, the sensor 360 may be amovement sensor (e.g., an accelerometer in a mobile phone that workssubstantially all the time), and the device 358 may be a batteryoperated EEG sensor embedded in headgear. In another example, the sensor360 may be a camera embedded in a television, such as a camera thatenables gesture control of the television and thus operatessubstantially whenever the television is on; the device 358 in thisexample may be a bracelet with embedded sensors that monitor heart rateand/or GSR. In one example, by “substantially independently of whetherthe user is exposed . . . ” it is meant that the normal behavior, unlessspecifically overridden by the user or software, is for the sensor 360to operate without respect to tags. In another example, by“substantially independently of whether the user is exposed . . . ” itis meant that the sensor 360 has a controller that does not take intoconsideration instructions or indications that appear in tags.

In one embodiment, the mode-selector 352 and/or the threshold module 362are part of a device belonging to the user 114. Optionally, the deviceis a mobile device, such as a smart phone or tablet computer.Optionally, the mode-selector 352, and/or the threshold module 362, areimplemented, at least in part, as software running on a processor of thedevice.

In another embodiment, the mode-selector 352 and/or the threshold module362 are implemented, at least in part, via a program that is run on aprocessor located on a server typically remote of the user, such as acloud-based server.

In one embodiment, the mode-selector 352 and the threshold module 362are implemented at least in part with software. Optionally, themode-selector 352 and the threshold module 362 share at least some ofthe same software modules. Optionally, the mode-selector 352 and thethreshold module 362 both utilize the same processor to run at leastsome of their software modules. Optionally, the mode-selector 352 andthe threshold module 362 are both implemented via the same program.

In one embodiment, the mode-selector 352 is implemented as part of thedevice 358. In one example, the mode-selector 352 is implemented as asoftware module associated with the device 358. In another example, themode-selector 352 is realized via hardware that is embedded in thedevice 358.

In one embodiment, the threshold module 362 is implemented as part ofthe sensor 360. In one example, threshold module 362 is implemented as asoftware module associated with the sensor 360. In another example, thethreshold module 362 is realized via hardware that is embedded in thesensor 360.

In one embodiment, a segment of content may include visual content, suchas images and/or video. A content delivery module may be used to exposethe segments to the user 114. Optionally, the device 358 has at leastone component that is not shared with the content delivery module.Additionally or alternatively, the device 358 may include at least onecomponent that is shared with the content delivery module. Additionallyor alternatively, the device 358 may be a battery powered sensor, andthe content delivery module and the battery powered sensor are realizedin the same device.

In one embodiment, the first tag 292 a and the second tag 292 b aregenerated by an interactive computer game which, such as the interactivecomputer game 294 illustrated in FIG. 5. The computer game may generatesegments of content to which the user may be exposed. Optionally, tagsare generated by the computer game in order to gauge affective responseof the user to segments of content generated by the computer game.

FIG. 9 illustrates one embodiment of a method for reducing powerconsumption by overriding an instruction to measure. The method includesthe following steps:

In step 370, receiving first and second tags corresponding to first andsecond segments of content, respectively; the first and second tagsindicate to operate a device in a first mode of operation to measureaffective response to the first and second segments.

In step 371, receiving a first measurement of a user taken by a sensoressentially before end of exposure of the user to the first segment.Alternatively or additionally, the first measurement may be taken duringa period that starts before the start of the exposure of the user to thefirst segment.

In step 372, determining that the first measurement reaches a firstpredetermined threshold, and selecting the first mode of operation forthe device to operate in order to measure affective response of the userto the first segment. Optionally, the device is also configured tooperate in the first mode of operation during a duration indicated bythe first tag.

In step 373, receiving a second measurement of the user, taken by thesensor essentially before end of exposure of the user to the secondsegment. Alternatively or additionally, the second measurement may betaken during a period that starts before the start of the exposure ofthe user to the second segment.

And In step 374, determining that the second measurement does not reacha second predetermined threshold, and overriding first mode of operationindicated by the second tag by selecting a second mode of operation inwhich to operate the device to measure affective response of the user tothe second segment. Optionally, the device is also configured to operatein the second mode of operation during a duration indicated by thesecond tag.

In one embodiment, power consumption of the device when operating in thesecond mode of operation is significantly lower than the powerconsumption of the device when operating in the first mode of operation.Thus, the overriding implemented in step 374 can enable the system toreduce its energy consumption. For example, at certain times, it mightnot be beneficial for the system to measure the user with the device;for instance, when the user is not paying attention to the content(e.g., not looking at the content), and/or the user is not in a desiredstate (e.g., a measurement taken by the sensor indicate the user is notin the desired emotional state). During such times, the system may electto override an instruction to measure the user, and thus reduce thepower consumed by the device without significantly reducing from theutility of the measurements of the device to the system (since in thisexample, they are not suitable).

In one embodiment, a system configured to reduce power consumption byoverriding an instruction to measure, comprising: a mode-selectorconfigured to receive first and second tags corresponding to first andsecond segments of content, respectively; wherein the first and secondtags indicate to operate a device in a first mode of operation tomeasure affective response to the first and second segments; and athreshold module configured to receive first and second measurements ofa user taken by a sensor essentially before end of exposure of the userto the first and second segments, respectively; the threshold module isfurther configured to determine that the first measurement reaches afirst predetermined threshold; the threshold module is furtherconfigured to determine that the second measurement does not reach asecond predetermined threshold, and to indicate thereof to themode-selector; the mode-selector is further configured to: operate thedevice in the first mode of operation to measure affective response ofthe user to the first segment, and to override first mode of operationindicated by the second tag, by operating the device in a second mode ofoperation to measure affective response of the user to the secondsegment; wherein power consumption of the device when operating in thesecond mode of operation is significantly lower than the powerconsumption of the device when operating in the first mode of operation.Optionally, the threshold module is further configured to determine thatthe first measurement reaches a first predetermined threshold, and toindicate thereof to the mode-selector. Optionally, the first tag furthercomprises a representation of an expected emotional response to thefirst segment. Optionally, the system further comprising a trainingmodule configured to train an emotional response predictor; the trainingmodule configured to receive the first tag and an affective responsemeasurement taken by the device while operating in the first mode ofoperation. Optionally, the sensor comprises an image capturing deviceand the sensor is a battery powered sensor that measures a physiologicalsignal and is attached to or implanted in the user. Optionally, thedevice is a battery powered sensor that is coupled to the user andmeasures a physiological signal of the user. Optionally, the sensor iscoupled to the user and measures a physiological signal of the user.Optionally, the sensor operates substantially independently of whetherthe user is exposed to segments having corresponding tags; and thedevice is mainly used to measure affective response of the user whileexposed to segments having corresponding tags. Optionally, the systemfurther comprising an interactive computer game configured to generatethe first segment and the first tag; wherein the interactive computergame has an element whose actions in the game are at least partiallycontrolled by the user. Optionally, the power consumption of the devicewhen operating in the second mode of operation is essentially zero.

In one embodiment, a method for reducing power consumption by overridingan instruction to measure, comprising: receiving first and second tagscorresponding to first and second segments of contents, respectively;wherein the first and second tags indicate to operate a device in afirst mode of operation to measure affective response to the first andsecond segments; receiving a first measurement of a user taken by asensor essentially before end of exposure of the user to the firstsegment; determining that the first measurement reaches a firstpredetermined threshold, and selecting the first mode of operation forthe device to operate in to measure affective response of the user tothe first segment; receiving a second measurement of the user taken bythe sensor essentially before end of exposure of the user to the secondsegment; and determining that the second measurement does not reach asecond predetermined threshold, and overriding first mode of operationindicated by the second tag by selecting a second mode of operation inwhich to operate the device to measure affective response of the user tothe second segment; wherein power consumption of the device whenoperating in the second mode of operation is significantly lower thanthe power consumption of the device when operating in the first mode ofoperation. Optionally, the first tag further comprises a representationof an expected emotional response of a user to being exposed to thefirst segment. Optionally, the method further comprises forwarding, to atraining module that trains an emotional response predictor, the firsttag and measurements of the affective response of the user to beingexposed to the first segment. Optionally, the first tag depends oninformation conveyed by the first segment, and does not depend mainly onbehavior of the user while being exposed to the first segment.Optionally, the sensor comprises an image capturing device and thesensor is a battery powered sensor that measures a physiological signaland is attached to or implanted in the user. Optionally, the device is abattery powered sensor that is coupled to the user and measures aphysiological signal of the user. Optionally, the sensor is coupled tothe user and measures a physiological signal of the user. Optionally,the sensor operates substantially independently of whether the user isexposed to segments having the corresponding tags; and the device ismainly used to measure affective response of the user while exposed tosegments of content having corresponding tags. Optionally, the methodfurther comprises generating the first segment and the first tag by aninteractive computer game having an element whose actions in the gameare at least partially controlled by the user; and the first tagcorresponds to a certain event occurring in the first segment.

In one embodiment, a system configured to reduce power consumption byoverriding instruction to measure, comprising: a mode-selectorconfigured to receive first and second tags corresponding to first andsecond segments of content, respectively; wherein the first and secondtags indicate to operate a device in a first mode of operation tomeasure affective response to the first and second segments; and whereinthe device is battery operated and measures a physiological signal of auser; and a threshold module configured to receive first and secondmeasurements of the user taken by a sensor essentially before end ofexposure of the user to the first and second segments, respectively; thethreshold module is further configured to determine that the firstmeasurement reaches a first predetermined threshold and indicate thereofto the mode-selector, and to determine that the second measurement doesnot reach a second predetermined threshold and indicate thereof to themode-selector; the mode-selector is further configured to: select thefirst mode of operation for the device to operate in to measureaffective response of the user to the first segment, and to overridefirst mode of operation indicated by the second tag, by selecting asecond mode of operation for the device to operate in to measureaffective response of the user to the second segment; wherein powerconsumption of the device when operating in the second mode of operationis significantly lower than the power consumption of the device whenoperating in the first mode of operation.

FIG. 10 illustrates one embodiment of a system 380 that is configured toreduce power consumption of a device 388 by overriding instructions tomeasure indicated by tags. The system 380 includes at least amode-selector 382 and a gaze analyzer 384.

The mode-selector 382 is configured to receive tags corresponding tosegments of content, such as the tags illustrated in FIG. 3. In oneembodiment, the mode-selector is configured to receive the first tag 292a and the second tag 292 b corresponding to first and second segments ofcontent, respectively. The first tag 292 a indicates to operate thedevice 388 in a first mode of operation 387 a to measure affectiveresponse to the first segment. The second tag 292 b indicates to operatethe device 388 in a first mode of operation 387 a to measure affectiveresponse to the second segment.

The gaze analyzer 384 is configured to receive, eye tracking data of theuser 114 acquired by an eye tracker 389, taken in period correspondingto an exposure of the user 114 to segments of content. Optionally, eyetracking data of the user 114 corresponding to a certain segment ofcontent is acquired essentially during exposure of the user to thesegment.

In one example, by “essentially during” it is meant that most of theduration during which the eye tracking data was acquired overlaps withwhen the user was exposed to the certain segment. In another example, by“essentially during” it is meant that the eye tracking data was acquiredduring the exposure, or during a period starting shortly before theexposure (e.g., shortly before may be up to 15 seconds before). Thus,the eye tracking data may reflect a condition of the user 114 during thetime of exposure to a segment of content, in particular it may bedetermined if the user is paying attention to the segment and/or adisplay via which the user is exposed, or is to be exposed, to thesegment.

In one embodiment, the gaze analyzer 384 is configured to receive firsteye tracking data of the user 114 acquired essentially during exposureof the user to the first segment. The gaze analyzer 384 is alsoconfigured to receive second eye tracking data of the user 114 acquiredessentially during exposure of the user to the second segment.Optionally, the first and second eye tracking data are acquired by theeye tracker 389. Optionally, the first and second eye tracking data mayundergo processing, such as filtration, normalization, and/or featureextraction. Optionally, processing the first and second eye trackingdata may involve analysis of the eye tracking data to determine fordifferent times directions of gaze of the user and/or objects the userwas looking at. Optionally, at least some of the processing of the eyetracking data is performed by gaze analyzer 384. Alternatively oradditionally, at least some of the processing is performed by the eyetracker 389. Additionally or alternatively, at least some of theprocessing is performed by an external module, such as a cloud-basedserver running analysis software. Additional information regarding eyetracking and computing the gaze-based attention level is given below.

The gaze analyzer 384 is also configured to determine from eye trackingdata it receives whether a gaze-based attention level, computed from theeye tracking data, reached a predetermined threshold. Optionally, thegaze analyzer 384 may indicate to other modules, such as themode-selector 382, whether the predetermined threshold was reached.Optionally, a gaze-based attention level, computed from eye trackingdata acquired during exposure of the user 114 to a segment of content,may correspond to an attention level of the user to the content.Optionally, this information may be utilized in order to determinewhether it is worthwhile at that time to measure the user with thedevice 388 in order to measure affective response of the user to thecontent: if the user is not paying attention to the content, then anaffective response measurement taken at that time may not accuratelyrepresent the user's attitude towards the content.

In one embodiment, the gaze analyzer 384 is configured to determine fromthe first eye tracking data that a gaze-based attention level to thefirst segment reaches a first predetermined threshold, and mayoptionally indicate to the mode-selector 382 thereof (i.e., that thegaze-based attention level to the first segment reaches the firstpredetermined threshold). Additionally, the gaze analyzer 384 isconfigured to determine from the second eye tracking data that agaze-based attention level to the second segment does not reach a secondpredetermined threshold, and may optionally indicate to themode-selector 382 thereof (i.e., that the gaze-based attention level tothe second segment does not reach the second predetermined threshold).

Herein, by stating that a value reaches a predetermined threshold, it ismeant that the value equals or exceeds the predetermined threshold. Inaddition, a predetermined threshold to which a value a measurement ofaffective response is compared, refers to a value of which there isprior knowledge. For example, the threshold value itself is known and/orcomputed prior to when the comparison is made. Additionally oralternatively, a predetermined threshold may utilize a threshold valuethat is computed according to logic (such as function) that is knownprior to when the comparison is made.

In one embodiment, the first predetermined threshold and the secondpredetermined threshold may represent different thresholds. For example,reaching the first predetermined threshold is based on achievingdifferent value than is required to pass the second predeterminedthreshold. Thus, there may be cases in which a certain value may reachthe first predetermined threshold, but not reach the secondpredetermined threshold. Alternatively, in another embodiment, the firstpredetermined threshold and the second predetermined threshold mayrepresent essentially the same predetermined threshold. Thus, in thisembodiment, a value that reaches the first predetermined threshold alsoreaches the second predetermined. Similarly, in this embodiment, a valuethat does not reach the first predetermined threshold also does notreach the second predetermined threshold.

In one embodiment, a predetermined threshold, such as the first orsecond predetermined thresholds, has a fixed value. For example, thepredetermined threshold may be 50%, representing that a user's eyetracking data should indicate that the user looked at the content for atleast half of the time. In another example, the predetermined thresholdmay be 7, and represent the fact that a user's gaze-based attentionlevel, as determined by a certain function that computes gazed-basedattention level from eye tracking data on a scale of 1 to 10, is 7 ormore.

In another embodiment, when a predetermined threshold is computedvarious factors may be taken into account, such as the length of theduration during which eye tracking data was acquired, the type ofcontent the use was exposed to, and/or characteristics of the userand/or the environment. Thus, based on one or more of these factors, thepredetermined threshold may represent lower or higher degrees ofrequired attention in order for the predetermined threshold to beconsidered reached. In one example, a longer duration in which eyetracking is acquired may lead to a decrease in a predetermined thresholdthat represents a percentage of time a user looks at the content; it maybe difficult in some situations to continually look at content (e.g.,when outside in the company of other people), so the threshold can berelaxed in such situations. In another example, the predeterminedthreshold used with a user that is a child may be lower than that of anadult, since for some children, it is more difficult to stayconcentrated for long. In another example, the predetermined thresholdused with content that is a commercial may be lower than thepredetermined threshold used for content that is a movie or televisionprogram, since users tend to pay less attention to commercials (thoughoften measurements in these situations are desired). In still anotherexample, the type of display on which content is consumed may influencethe predetermined threshold. For instance, a display that ishead-mounted (e.g., an augmented reality display) may have a different,possibly higher, predetermined threshold associated with it, compared toa display on a smartphone, since content on an augmented reality displaymay be clearer and easy to see if the user desires to look at it.

The gaze analyzer 384 may indicates to the mode-selector 382 whether agaze-based attention level computed from eye tracking data reaches apredetermined threshold. There may be different ways in which the gazeanalyzer 384 may make such indications. In one example, the gazeanalyzer 384 signals (e.g., via a message communicated to themode-selector 382) a value that describes if the gaze-based attentionlevel reached the threshold, and/or the actual gaze-based attentionlevel and/or the difference between the gaze-based attention level andthe threshold. In another example, the gaze analyzer 384 sends a signalto the mode-selector 382 when the gaze-based attention level reaches thethreshold, and if no signal is sent, the mode-selector 382 operatesunder the assumption that the gaze-based attention did not reach thethreshold. Alternatively, the gaze analyzer 384 may send a signal to themode-selector 382 when the gaze-based attention does not reach thethreshold, and if no signal is sent, the mode-selector 382 operatesunder the assumption that the gaze-based attention reached thethreshold.

The mode-selector 382 is configured to select a mode of operation forthe device 388 to operate in when measuring affective response of theuser 114. Optionally, the selection of a mode of operation may be doneaccording to an indication in a received tag. Optionally, themode-selector 382 may make a selection of the mode of operation alsoaccording to other information such as an indication from the gazeanalyzer 384 regarding whether or not a gaze-based attention levelcomputed from eye tracking data reaches a predetermined threshold.Optionally, a mode of operation for the device 388 is chosen from a setthat includes the first mode of operation 387 a and a second mode ofoperation 387 b.

In one embodiment, the mode-selector 382 selects the first mode ofoperation 387 a, as indicated by the first tag 292 a, for measuring theaffective response of the user to the first segment of content. Themode-selector 382 makes this selection because the gaze analyzer 384indicates that the gaze-based attention level to the first segmentreaches the first predetermined threshold. However, the mode-selector382 overrides the first mode of operation 387 a, indicated by the secondtag 292 b, based on indication of the gaze analyzer 384 that the secondthe gaze-based attention level to the second segment does not reach thesecond predetermined threshold. The power consumption of the device 388when operating in the second mode of operation 387 b is significantlylower than the power consumption of the device 388 when operating in thefirst mode of operation 387 a. Thus, by switching modes of operation ofthe device 388, the mode-selector 382 is able to help reduce the powerconsumption of the device 388 when it is not likely that measurements ofthe user with the device 388 are going to be useful.

In one example, the mode-selector 382 may override a tentative choice ofthe first mode of operation 387 a (as indicated by a tag), by replacingthe selected first mode of operation 387 a with the second mode ofoperation 387 b. For instance, this may be done by changing parametersoriginally set by the mode-selector 382 according to informationreceived from the threshold-module 384. In another example, overridingmay be achieved by the gaze analyzer 384 running before themode-selector 382 receives a tag indicating the desired mode ofoperation. If the gaze analyzer 384 determines that a measurement of theuser taken with the sensor 360 does not reach a predetermined threshold,it may prevent the tag from reaching the mode-selector 382, thus casingthe mode-selector 382 not to select the first mode of operation 387 afor the device 388. In this example, the default behavior of the device388 may be to operate in the second mode of operation 387 b; thusblocking the tag from reaching the mode-selector 382 and/or blocking themode-selector 382 from acting according to the tag is equivalent toselecting the second mode of operation for the device 388.

In some embodiments, the power consumption of the device 388 whileoperating in the first mode of operation 387 a may be significantlyhigher than its power consumption while operating in the second mode ofoperation 387 b. Consequently, by selecting for certain durations, thesecond mode of operation 387 b for the device 388 to operate in, insteadof the intended first mode of operation 387 a, the total amount ofenergy consumed over time by the device 388 may be reduced.Additionally, the mode-selector 382 may reduce the power consumption ofother elements that participate in the processing of the affectiveresponse measurements taken by the device 388, such as a processor thatprocesses the data comprising the measurements. Optionally, the powerconsumption of the device 388 while operating at the second mode ofoperation 387 b is essentially zero. For example, by essentially zero itis meant that the power consumption is at most 5% of the powerconsumption of the device 388 when operating at the first mode ofoperation 387 a.

In one embodiment, the mode-selector 382 selects a mode of operation,from two modes of operation: a first mode of operation that is a standbymode of operation, and a second mode of operation that is a normal modeof operation. In this example, as is the case with many sensors and/ordevices, when the device 388 operates in the standby mode, it consumessignificantly less power compared to when it operates in the normal modeof operation.

In another embodiment, selecting a mode of operation, by themode-selector 382, may involve selecting different parameter settingsfor the operation of a device, such as setting resolution of imagescaptured by a camera, or brainwave frequencies that are to be monitoredusing EEG.

In one embodiment, a mode of operation for a sensor, such as the modes387 a and/or 387 b, may be implicitly or explicitly selected. Forexample, an implicit selection of the mode of operation may occur if thedevice 388 does not receive a signal that may indicate and/or influencea mode of operation at which it operates. By contrast, an explicitselection of the mode of operation may involve sending the device 388information that indicates which mode of operation to operate in and/orparameter values that determine how the device 388 is to operate.

In one embodiment, the system 380 includes a measurement EmotionalResponse Predictor (measurement ERP) configured to predict the user'semotional response from data comprising measurements of affectiveresponse of the user 114 taken with the device 388. Optionally, themeasurements taken by the device are processed prior to being sent tothe measurement ERP for prediction. For example, the measurements mayundergo filtering and/or feature extraction.

In one embodiment, the system 380 includes a model trainer 290, asillustrated in FIG. Optionally, the model trainer 290 may receive asinput measurements taken by the device 388, in temporal vicinity of thefirst duration indicated by the tag 292 a. The measurements may includeaffective response of the user 114 to the first segment of contentcorresponding to the tag 292 a.

In one embodiment, the computer game 294, which is illustrated in FIG.5, generates the tags 292 a and 292 b received by the mode-selector 382.Measurements taken with the device 388 during the first duration,indicated by the tag 292 a, may then be utilized by the interactivecomputer game in order to determine emotional response of the user 114(e.g., by providing the measurements to a measurement ERP). Thepredicted emotional response may be used in order to determine ifelements in the game need to be changed in order to improve the userexperience.

In one embodiment, the functionality of the system 380 is included inthe interactive computer game 294. In this embodiment, the computer gamesets a mode of operation for the device 388. For example, the system 380is implemented, at least in part, as software running on a processorbelonging to a game console. In another example, the system 380 isimplemented, at least in part, as software running on a server running agame such as an online multiplayer game.

In one embodiment, the first and second eye tracking data are acquiredutilizing an eye tracker, such as the eye tracker 389, which includesimage capturing device. Optionally, the device 388 may be a batterypowered sensor that measures a physiological signal and is coupled tothe user. Optionally, the first and second eye tracking data arecollected substantially independently of whether the user is exposed tosegments of content identified by tags, and the device 388 comprises abattery powered sensor that is mainly used to measure affective responseof the user while exposed to segments identified by tags.

In one embodiment, the eye tracker 389 operates substantiallyindependently of whether the user 114 is exposed to segments of contentidentified by tags. Optionally, the device 388 is a battery poweredsensor that is mainly used to measure affective response of the userwhile exposed to segments identified by tags. In one example, by“substantially independently of whether the user is exposed . . . ” itis meant that the normal behavior, unless specifically overridden by theuser or software, is for the eye tracker 389 to operate without respectto tags. In another example, “substantially independently of whether theuser is exposed . . . ” it is meant that the eye tracker 389 has acontroller that may not necessarily take into consideration instructionsor indications that appear in tags.

In one embodiment, a segment of content may include visual content, suchas images and/or video. A content delivery module may be used to exposethe segments to the user 114. Optionally, the device 388 has at leastone component that is not shared with the content delivery module.Additionally or alternatively, the device 388 may include at least onecomponent that is shared with the content delivery module. Additionallyor alternatively, the device 388 may be a battery powered sensor, andthe content delivery module and the battery powered sensor are realizedin the same device.

In one embodiment, the mode-selector 382 and/or the gaze analyzer 384are part of a device belonging to the user 114. Optionally, the deviceis a mobile device, such as a smart phone or tablet computer.Optionally, the mode-selector 382, and/or the gaze analyzer 384, areimplemented, at least in part, as software running on a processor of thedevice.

In another embodiment, the mode-selector 382 and/or the gaze analyzer384 are implemented, at least in part, via a program that is run on aprocessor located on a server typically remote of the user, such as acloud-based server.

In one embodiment, the mode-selector 382 and the gaze analyzer 384 areimplemented at least in part with software. Optionally, themode-selector 382 and the gaze analyzer 384 share at least some of thesame software modules. Optionally, the mode-selector 382 and the gazeanalyzer 384 both utilize the same processor to run at least some oftheir software modules. Optionally, the mode-selector 382 and the gazeanalyzer 384 are both implemented via the same program.

In one embodiment, the mode-selector 382 is implemented as part of thedevice 388. In one example, the mode-selector 382 is implemented as asoftware module associated with the device 388. In another example, themode-selector 382 is realized via hardware that is embedded in thedevice 388.

In one embodiment, the gaze analyzer 384 is implemented as part of theeye tracker 389. In one example, gaze analyzer 384 is implemented as asoftware module associated with the eye tracker 389. In another example,the gaze analyzer 384 is realized via hardware that is embedded in theeye tracker 389.

In one embodiment, the system 380 includes a measurement EmotionalResponse Predictor (measurement ERP) configured to predict the user'semotional response from data comprising measurements of affectiveresponse of the user 114 taken with the device 388. Optionally, themeasurements taken by the device are processed prior to being sent tothe measurement ERP for prediction. For example, the measurements mayundergo filtering and/or feature extraction.

In one embodiment, the system 380 includes a model trainer 290, asillustrated in FIG. Optionally, the model trainer 290 may receive asinput measurements taken by the device 388, in temporal vicinity of thefirst duration indicated by the tag 292 a. The measurements may includeaffective response of the user 114 to the first segment of contentcorresponding to the tag 292 a.

FIG. 11 illustrates one embodiment of a method for reducing powerconsumption by utilizing eye tracking to override instructions conveyedby tags. The method includes the following steps:

In step 390, receiving first and second tags corresponding to first andsecond segments of contents, respectively; the first tag indicates tooperate a device in a first mode of operation to measure affectiveresponse to the first segment, and the second tag indicates to operatethe device in the first mode to measure affective response to the secondsegment.

In step 391, receiving first eye tracking data of the user acquiredessentially during exposure of the user to the first segment.

In step 392, determining, based on the first eye tracking data, that agaze-based attention level to the first segment reaches a firstpredetermined threshold, and operating the device in the first mode ofoperation to measure affective response of the user to the firstsegment.

In step 393, receiving second eye tracking data of the user acquiredessentially during exposure of the user to the second segment.

And in step 394, determining, based on the second eye tracking data,that a gaze-based attention level to the second segment does not reach asecond predetermined threshold, and overriding first mode of operationindicated by the second tag by operating the device in a second mode ofoperation to measure affective response of the user to the secondsegment.

In one embodiment, power consumption of the device when operating in thesecond mode of operation is significantly lower than the powerconsumption of the device when operating in the first mode of operation.Thus, the overriding implemented in step 394 can enable the system toreduce its energy consumption. For example, at certain times, it mightnot be beneficial for the system to measure the user with the device;for instance, when the user is not paying attention to the content(e.g., not looking at the content). During such times, the system mayelect to override an instruction to measure the user, and thus reducethe power consumed by the device without significantly reducing from theutility of the measurements of the device to the system (since in thisexample, they are not suitable).

In one embodiment, a system configured to reduce power consumption byutilizing eye tracking to override instructions conveyed by tags,comprising: a mode-selector configured to receive first and second tagscorresponding to first and second segments of content, respectively;wherein the first and second tags indicate to operate a device in afirst mode of operation to measure affective response to the first andsecond segments; and a gaze analyzer configured to receive first andsecond eye tracking data of a user acquired essentially during exposureof the user to the first and second segments, respectively; the gazeanalyzer is further configured to determine from the first eye trackingdata that a gaze-based attention level to the first segment reaches afirst predetermined threshold; the gaze analyzer is further configuredto determine from the second eye tracking data that a gaze-basedattention level to the second segment does not reach a secondpredetermined threshold, and to indicate thereof to the mode-selector;the mode-selector is further configured to: operate the device in thefirst mode of operation to measure affective response of the user to thefirst segment, and to override first mode of operation indicated by thesecond tag, by operating the device in a second mode of operation tomeasure affective response of the user to the second segment; whereinpower consumption of the device when operating in the second mode ofoperation is significantly lower than the power consumption of thedevice when operating in the first mode of operation. Optionally, thegaze analyzer is further configured to indicate to the mode-selectorthat the gaze-based attention level to the first segment reaches thefirst predetermined threshold. Optionally, the first and second tagsfurther comprise representations of expected emotional responses to thesegments. Optionally, the system further comprises a training moduleconfigured to receive the first tag and measurements taken by the deviceduring the first duration, and to train an emotional response modelutilizing the first tag and the measurements. Optionally, the first andsecond eye tracking data are acquired utilizing an image capturingdevice, and the device is a battery powered sensor that measures aphysiological signal and is coupled to the user. Optionally, the deviceis a battery powered sensor that is coupled to the user and measures aphysiological signal. Optionally, the first and second eye tracking dataare collected substantially independently of whether the user is exposedto segments of content identified by tags; and the device is a batterypowered sensor that is mainly used to measure affective response of theuser while exposed to segments identified by tags. Optionally, thesystem further comprises an interactive computer game configured togenerate the first and second segments and the first and second tags;wherein the interactive computer game has an element whose actions inthe game are at least partially controlled by the user. Optionally, thepower consumption of the device when operating in the second mode ofoperation is essentially zero.

In one embodiment, a method for reducing power consumption by utilizingeye tracking to override instructions conveyed by tags, comprising:receiving first and second tags corresponding to first and secondsegments of content, respectively; wherein the first and second tagsindicate to operate a device in a first mode of operation to measureaffective response to the first and second segments; receiving first eyetracking data of the user acquired essentially during exposure of theuser to the first segment; determining, based on the first eye trackingdata, that a gaze-based attention level to the first segment reaches afirst predetermined threshold, and operating the device in the firstmode of operation to measure affective response of the user to the firstsegment; receiving second eye tracking data of the user acquiredessentially during exposure of the user to the second segment; anddetermining, based on the second eye tracking data, that a gaze-basedattention level to the second segment does not reach a secondpredetermined threshold, and overriding first mode of operationindicated by the second tag by operating the device in a second mode ofoperation to measure affective response of the user to the secondsegment; wherein power consumption of the device when operating in thesecond mode of operation is significantly lower than the powerconsumption of the device when operating in the first mode of operation.Optionally, the first tag further comprises a representation of anexpected emotional response of a user to being exposed to the firstsegment. Optionally, the method further comprises forwarding, to atraining module that trains an emotional response predictor, the firsttag and measurements of the affective response of the user to beingexposed to the first segment. Optionally, the first tag depends oninformation conveyed by the first segment, and does not depend mainly onbehavior of the user while being exposed to the first segment.Optionally, the first and second eye tracking data are acquiredutilizing an image capturing device and the device is a battery poweredsensor that measures a physiological signal and is coupled to the user.Optionally, the device is a battery powered sensor that is coupled tothe user, and measures a physiological signal. Optionally, the first andsecond eye tracking data are collected substantially independently ofwhether the user is exposed to segments of content identified by tags;and the device is a battery powered sensor that is mainly used tomeasure affective response of the user while exposed to segmentsidentified by tags. Optionally, the method further comprises generatingthe first segment and the first tag by an interactive computer gamehaving an element whose actions in the game is at least partiallycontrolled by the user; wherein the first tag corresponds to a certainevent occurring in the first segment.

In one embodiment, a system configured to reduce power consumption byutilizing eye tracking to override instructions conveyed by tags,comprising: a mode-selector configured to receive first and second tagscorresponding to first and second segments of content, respectively;wherein the first and second tags indicate to operate a device in afirst mode of operation to measure affective response to the first andsecond segments; and a gaze analyzer configured to receive first andsecond eye tracking data of a user acquired essentially during exposureof the user to the first and second segments, respectively; gazeanalyzer is further configured to determine from the first eye trackingdata that a gaze-based attention level to the first segment reaches afirst predetermined threshold, and to indicate thereof to themode-selector; the mode-selector is further configured to select thefirst mode of operation for the device to operate in to measureaffective response of the user to the first segment; the gaze analyzeris further configured to determine from the second eye tracking datathat a gaze-based attention level to the second segment does not reach asecond predetermined threshold, and to indicate thereof to themode-selector; the mode-selector is further configured to override firstmode of operation indicated by the second tag by selecting a second modeof operation for the device to operate in to measure affective responseof the user to the second segment; wherein power consumption of thedevice when operating in the second mode of operation is significantlylower than the power consumption of the device when operating in thefirst mode of operation. Optionally, the first and second eye trackingdata are acquired utilizing an image capturing device and the device isa battery powered sensor that measures a physiological signal and iscoupled to the user. Optionally, the image capturing device and thebattery powered sensor utilize separate power sources.

FIG. 12 illustrates one embodiment of a system 100 configured to reducepower consumption according to analysis of content. The system 100includes at least a content ERA 104 and a controller 108.

The content ERA (Emotional Response Analyzer) 104 is configured to (i)receive a first segment 102 a of content and a second segment 102 b ofcontent, (ii) analyze the segments 102 a and 102 b, and (iii) output afirst indication 106 a and a second indication 106 b. The firstindication 106 a indicates that a first value related to a prediction ofemotional response to the first segment 102 a does not reach a firstpredetermined threshold. The second indication 106 b indicates that asecond value related to a prediction of emotional response to the secondsegment 102 b does reach a second predetermined threshold.

Herein, a first segment of content and a second segment of content, suchas the first segment 102 a and the second segment 102 b, are notnecessarily segments of the same content item. For example, the segment102 a may be a clip from a first movie, and the segment 102 b may be aclip of a second, different, movie. In another example, the firstsegment 102 a is a portion of a video game play, and the second segment102 b is a different type of content, such as a web page. Optionally,the first and second segments are different segments of the samecontent. For example, the first segment 102 a and the second segment 102b may be first and second scenes in the same movie.

A segment of content, such as the segments 102 a and/or 102 b, is aportion of content to which a user 114 may be exposed. For example, thesegment of content may be a portion of a conversation (e.g., messagesconveyed in text, audio, video, and/or tactile stimulation), a portionof visual media content (e.g., belonging to a movie, web site, and/orcomputer game), and/or a portion of audio content (e.g., music and/orsound effects). Optionally, an interaction of the user 114 with adigital device may lead to the generation of a segment of content. Forexample, an interaction of the user with a computer gaming device maylead to the generation of a segment of gaming content such as the userbattling a monster in a virtual world. Optionally, an interaction of theuser 114 with a digital device may lead to the presentation of anessentially already prepared segment of content. For example, aninteraction of the user with a browser on a tablet computer can triggerthe presentation of a video clip to the user on the tablet's display.

In one embodiment, a segment of content might undergo processing beforebeing received by the content ERA 104 and/or before the content ERA 104analyzes the segment of content. Thus, the data representing the segmentof content, which gets analyzed by the content ERA 104, may not be thesame data presented to the user 114. For example, a segment of contentthat includes video may undergo feature extraction prior to being sentto the content ERA 104 (e.g., in order to identify object, people,and/or the genre). In this example, the exposure of the user 114 to thesegment of content involves viewing a video clip. However, the contentERA 104 receives related data, such as feature values corresponding toobjects, people, and/or the genre of the video, and performs itsanalysis on that data.

In one embodiment, the content ERA 104 may output the first indication106 a and the second indication 106 b, which may be conveyed to thecontroller 108. The controller 108 is configured to select a mode ofoperation for operating a device 112 based on a received indication.Optionally, the device 112 includes a sensor that may be used to measureaffective response of the user 114. In one example, based on theindication 106 a, the controller 108 is configured to select for thedevice 112 a first mode operation 110 a, and based on the secondindication 106 b, the controller 108 is configured to select for thedevice 112 a second mode of operation 110 b. Optionally, while operatingin the first mode of operation 110 a, the device 112 measures theaffective response of the user 114 to the first segment 102 a.Optionally, while operating in the second mode of operation 110 b, thedevice 112 measures the affective response of the user 114 to the secondsegment 102 b. Optionally, the indications 106 a and 106 b may bereceived separately by the controller 108 (e.g., each indication iscommunicated in a separate communication), or may be received jointly(e.g., both indications are communicated in the same communication).

In one embodiment, the content ERA 104 utilizes a prediction ofemotional response to a segment of content, and provides in acorresponding indication it produces one or more values related to theprediction. Optionally, to make the prediction, the content ERA utilizesa personalized model that provides indications optimized for the user114. Alternatively, to make the prediction, the content ERA utilizes ageneral model that is not personalized for the user 114.

In another embodiment, the indication 106 a and/or the indication 106 bmay be interpreted as operating instructions for another module in thesystem, such as the controller 108 and/or the device 112. For example,the indications may be interpreted as instructions to turn the device112 on or off, and/or to select a specific mode of operation for thedevice 112. For example, the indication first 106 a may be interpretedas an instruction to select the first mode 110 a, and similarly, thesecond indication 106 b may be interpreted as an instruction to selectthe second mode 110 b. In another example, the indications 106 a and/or106 b may indicate how long the device 112 should operate and/or specifya period of time (e.g., by providing start and end times) in which adevice 112 should operate.

It is noted that phrases like “to indicate something” or “indicative ofsomething” are to be interpreted as directly and/or indirectly signalingthe something. Therefore, a phrase like “the indication is indicative ofreaching a threshold”, means that the indication directly or indirectlysignals that the threshold is reached. For example, to indicatesomething the indication may explicitly state it (e.g., includeinformation that may be interpreted as reaching or not reaching athreshold). In another example, to indicate something the indication mayinclude a value, which when interpreted by a recipient conveys thesignal (e.g., an indication may include a value of heart rate which maybe compared to a certain heart rate value to determine if the thresholdis reached). In yet another example, a property of the indication (e.g.,its size, type, and/or the fact that it is sent), are sufficient tosignal something to a recipient.

In some embodiments, the first indication 106 a indicates whether afirst value, related to a prediction of emotional response to the firstsegment 102 a, reaches a first predetermined threshold. Additionally,the second indication 106 b may indicate whether a second value, relatedto a prediction of emotional response to the second segment 102 b,reaches a second predetermined threshold. Optionally, the prediction ofemotional response to the first segment 102 a is generated as part ofthe analysis of the first segment 102 a by the content ERA 104.Optionally, the prediction of emotional response to the second segment102 b is generated as part of the analysis of the second segment 102 bby the content ERA 104. Optionally, a content emotional responsepredictor (content ERP) is utilized to make the prediction of emotionalresponse to the first segment 102 a and/or the prediction of emotionalresponse to the second segment 102 b.

Herein, by stating that a value reaches a predetermined threshold, it ismeant that the value equals or exceeds the predetermined threshold. Inaddition, a predetermined threshold to which a value related to aprediction of emotional response is compared, refers to a thresholdvalue of which there is prior knowledge. In one example, the thresholdvalue itself is known and/or computed prior to when the comparison ismade. Alternatively or additionally, a predetermined threshold may be athreshold that is computed according to logic (such as function) that isknown prior to when the comparison is made.

In one example, the first predetermined threshold and the secondpredetermined threshold represent different thresholds. For example,reaching the first threshold is based on achieving different value thanis required to pass the second predetermined threshold. Thus, a valuerelated to a predicted emotional response to a certain segment ofcontent might reach the first predetermined threshold but not reach thesecond predetermined threshold, or vice versa. In another example, thefirst predetermined threshold and the second predetermined thresholdrepresent the same threshold. Thus, any value related to a predictedemotional response to a certain segment that reaches the firstpredetermined threshold also reaches the second predetermined.Similarly, any value related to a predicted emotional response to acertain segment that does not reach the first predetermined thresholdalso does not reach the second predetermined threshold.

In one embodiment, the value related to the prediction of emotionalresponse to a segment of content may represent a type and/or extent ofan emotional response to the segment of content. For example, thepredicted emotional response needs to be of a certain type and/or be ata certain extent, in order for the predetermined threshold to bereached.

In one example, a predetermined threshold may relate to an emotionalresponse of happiness; thus, if a user is not predicted to be happy fromexposure to the first segment 102 a, the indication 106 a may indicatethat the first predetermined threshold is not reached. However, if theuser is predicted to feel happy due to being exposed to the secondsegment 102 b, the indication 106 b may indicate that the secondpredetermined threshold is reached. Optionally, the user for which thepredictions of emotional response are made is the user 114.

In another example, a predetermined threshold may relate to the extentof expected expression of excitement. The expected excitement to asegment of content may be a predicted value on a scale of 1 to 10; apredetermined threshold, in this example, may be set to a level of 5 onthe scale of 1 to 10. Thus, if a user is predicted to have an excitementof 5 to a certain segment, the indication may indicate that thepredetermined threshold is reached, while if the excitement is predictedto be at level 2, the indication may indicate that the predeterminedthreshold is not reached.

A predetermined threshold, such as the first and/or second predeterminedthresholds, may be set based on a property of a segment of contentand/or analysis of a segment of content. For example, the type ofcontent may influence the value of the threshold. Given a segment ofcontent that is a from a video game, a first value representingexcitement may be used as a threshold, while if the segment comes from atelevision program, a second, possibly lower, value may be used. In thisexample, the fact that a user is likely to be more emotionally involvedin a video game (which is typically interactive), leads to a higherthreshold for a segment of a game compared to the threshold that is usedfor the video game. In another example, a short segment of content(e.g., a 10 second video clip) corresponds to a lower threshold valuefor user excitement than a longer segment of content (e.g., a 2 minutevideo clip). In this example, it is assumed that the longer a user isexposed to content (at least up to a certain point), the stronger theuser's emotional response is likely to be.

In some embodiments, a device, such as the device 112, is used tomeasure affective response of the user 114. Optionally, the deviceincludes a sensor and/or is a sensor. Optionally, the sensor may be aphysiological sensor (e.g., a sensor that measures heart rate, galvanicskin response, and/or brainwave activity), and/or a sensor that measuresthe user's behavior (e.g., a camera, and/or a motion detector).Optionally, the device may include additional components to the sensor,such as a memory, a processor, a battery, a transmitter, and/or areceiver. Optionally, the device may be coupled to a user. Herein aphrase like “a device coupled to a user” refers to a device that isattached to the user (e.g., on clothing, a bracelet, headgear), incontact with the user's body (e.g., a sticker on a user's skin), and/orimplanted in the user's body (e.g., an implanted heart-rate sensor orimplanted EEG electrodes).

In some embodiments, the device 112 may output measurements 113 of theaffective response of the user 114. Optionally, the measurements 113 areforwarded to other modules of the system such as the measurementEmotional Response Predictor (measurement ERP) 118. Optionally, some ofthe measurements 113 of the device 112 may include raw measurementvalues (i.e., the values undergo very little transformation prior totheir output). Alternatively or additionally, the measurements 113 mayinclude processed values such as values that are the result offiltering, normalization, signal processing, feature extraction,encryption, and/or compression.

In some embodiments, a mode of operation selected by a controller, suchas the first mode 110 a and/or the second mode 110 b, may have presetoperation parameters (such as voltage, resolution, sampling rate).Alternatively or additionally, the mode of operation may include certainoperation parameters that are computed by the controller according tothe received indication and/or other data that is available at the timeof computation.

In some embodiments, a mode of operation for the device 112 is selectedfrom a set comprising at the first mode 110 a and the second mode 110 b.Optionally, the power consumption of the device 112 while operating inthe first mode 110 a is significantly lower than its power consumptionwhile operating in the second mode 110 b. Consequently, by selecting themode of operation with respect to the information in an indicationproduced by the content ERA 104, the system 100 may reduce the amount ofpower consumed over time by the device 112 for measuring the affectiveresponse of the user 114. Additionally, the system 100 may reduce thepower consumption of other elements that participate in the processingof the affective response measurements taken by the device 112, such asa processor that processes the data comprising the measurements, and/ora transmitter that transmits the measurements. Optionally, the powerconsumption of the device 112 while operating in the first mode ofoperation is substantially zero. In one example, power consumption thedevice 112 is considered to be substantially zero if it is less than 10%of the power consumption of the device 112 when it is operating in thesecond mode of operation 110 b. In another example, power consumptionthe device 112 is considered to be substantially zero if it is less thandouble the of power consumption of the device 112 when the device 112 isin a standby mode in which the device does not take measurements, doesnot transmit measurements, and/or does not process measurements.

In one example, the controller 108 chooses a mode of operation for thedevice 112, from two modes of operation: the first mode 110 acorresponds to a standby mode of operation and the second mode 110 bcorresponds to a normal mode of operation. In this example, as is thecase with many devices, when the device 112 operates in the standbymode, it consumes significantly less power compared to when it operatesin the normal mode of operation.

In another example, an indication generated by the content ERA 104, suchas the first indication 106 a and/or the second indication 106 b, mayinclude a value that is indicative of a predicted interest level of auser in a segment of content. The controller 108 may select, accordingto the predicted interest level, parameters that define a mode ofoperation of the device 112, such as a resolution of images captured bya camera that is part of the device 112. If the indication indicatesthat the predicted interest level is low, the controller 108 may selecta first mode of operation in which a camera captures low resolutionimages (e.g., 320×240 pixels). However, if the indication indicates thepredicted interest level is high, the controller 108 may select a secondmode of operation in which the camera captures high-resolution images(e.g., 1920×1080 pixels). In some cases, different resolutions lead todifferent power consumptions for acquiring images. For instance, thelower resolution of the first mode of operation can cause a camera toutilize significantly less CMOS gates, compared to the number of CMOSgates it needs to utilize in the second mode of operation. Thus, in thisexample, the power consumption of the device 112 operating in the firstmode of operation is significantly lower than the power consumption ofthe device 112 operating in the second mode of operation.

In one embodiment, as part of its analysis of a segment of content, thecontent ERA 104 predicts a value describing the expected emotionalresponse of a user to a segment of content. Such a value may be includedin an indication created by the content ERA 104, which is conveyed tothe controller 108. Additionally or alternatively, an indication may beindicative of whether the expected emotional response reaches apredetermined threshold; and the controller 108 selects a mode ofoperation based on whether or not the predetermined threshold isreached.

For example, in the course of its analysis, the content ERA 104 predictshow frightened a user is likely to get from viewing a segment of contentthat is a video clip. An indication that is generated by the content ERA104, indicates on a scale of 1-10 how frightened a user is expected tobe (10 being the extremely frightened). The controller 108 receives theindication, and with respect to the indication, selects a mode ofoperation for the device 112, which may be an EEG head-mounted, batteryoperated, sensor that may be used to measure the affective response ofthe user 114 while viewing the video clip. In order to save power, thesystem may elect to measure the affective response of the user with theEEG sensor, while the clip is played, if the emotional response isexpected to reach a predetermined threshold level, such as 3 on thescale of 1-10. Measuring the user with the EEG may help determine towhat extent the user was really frightened while watching the videoclip; however, if the predicted value is too low, it is not likely thatthe clip is going to scare the user at all, so the system chooses not towaste power on confirming that. Thus, if the expected emotional responseis below the threshold, the controller 108 may select a “low power” modeof operation for the device 112, in which the device 112 consumes verylittle power. However, if the expected emotional response reaches thethreshold, the controller 108 may select a “regular” mode of operationfor the device 112, in which the device 112 consumes significantly morepower than in the low power mode.

In one embodiment, the controller 108 may be implemented as part of thecontent ERA 104. In another example, the controller 108, the device 112,and the content ERA 104 are combined in a single device. Optionally, thedevice is battery powered (e.g., a smart phone).

In one embodiment, a segment of content may include visual content, suchas images and/or video. A content delivery module may be used to exposethe segments to the user 114. Optionally, the device 112 has at leastone component that is not shared with the content delivery module.Additionally or alternatively, the device 112 may include at least onecomponent that is shared with the content delivery module. Additionallyor alternatively, the device 112 may be a battery powered sensor, andthe content delivery module and the battery powered sensor are realizedin the same device.

In one embodiment, at least one element of the controller 108 isimplemented as a dedicated hardware element. For example, the controller108 may be a component coupled to the device 112. In another example,the controller 108 is coupled to a processor that processes affectiveresponse measurements.

In one embodiment, the controller 108 is implemented in software runningon a general purpose processor. In some cases, the controller may run,at least in part, on a device belonging to the user, such as a mobilephone. In some cases, the controller may run, at least in part, on asystem remote of the user, such as on a cloud-based server.

In one embodiment, the controller 108 that selects a mode of operationfor the device 112 is realized as hardware that is embedded in thehousing of the device 112. For example, the controller for a mobile EEGmeasuring headset may be embedded in the headset itself, and optionallycommunicate data wirelessly.

In another embodiment, the controller 108 is realized as software thatruns on a processor that belongs to the device 112. For example, thecontroller for a mobile EEG measuring headset may run on a processorthat belongs to the headset and is used to process EEG measurements.

In one embodiment, the controller 108 and/or the content ERA 104 areimplemented at least in part, as software running on a remote processor,such as a cloud-based server. For example, the cloud-based service canbe used to analyze content that is to be presented to the user 114, suchas video to be streamed to the user's display. According to indicationsof the content ERA's analysis, the controller 108 running on thecloud-based service may send different operating instructions to thedevice 112 regarding how to measure the user.

FIG. 14 illustrates one embodiment of a method for reducing powerconsumed measuring affective response. The method includes the followingsteps:

In step 130, receiving a first indication derived from analysis of afirst segment of content.

In step 131, determining that the first indication indicates that afirst value related to a prediction of emotional response to the firstsegment does not reach a first predetermined threshold.

In step 132, configuring a device, based on the first indication, tooperate in a first mode of operation to measure affective response of auser to the first segment.

In step 133, receiving a second indication derived from analysis of asecond segment of content.

In step 134, determining that the second indication indicates that asecond value related to a prediction of emotional response to the secondsegment does reach a second predetermined threshold.

And in step 135, configuring the device, based on the second indication,to operate in a second mode of operation to measure affective responseof the user to the second segment.

In one embodiment, power consumed by the device, per unit of measurementtime, for measuring the affective response of the user to the firstsegment, is significantly lower than power consumed by the device formeasuring the affective response of the user to the second segment.

It is noted that references to first and second indices, such as thefirst and second segments, the first and second indications, and/or thefirst and second modes of operation, are not intended to limit the orderof presenting the enumerated elements in any way. For example, thesecond indication may be received before the first indication, and theindications may also include a third indication related to a thirdpredetermined threshold.

In one embodiment, the first and second segments are received from aninteractive computer game module. The module provides informationregarding an expected emotional response to the second segment, which isused in order to generate the second indication. Additionally, thecomputer game may also provide information regarding an expectedemotional response to the first segment, which is used to generate thefirst indication.

FIG. 15 is a schematic illustration of an embodiment comprising a systemthat includes a computer game 120 that outputs segments of content 102 aand 102 b, and information 122 about the expected emotional response tothe segments of content 102 a and/or 102 b. Optionally, the user 114provides the interactive computer game 120 with input that influencesevents and/or characters in the computer game. Optionally, the segments102 a and 102 b are displayed to the user 114 and/or are provided to thecontent ERA 104, possibly after undergoing some form of processing(e.g., feature extraction). Optionally, the information 122 about theexpected emotional response is provided to the content ERA 104 in orderto assist with the analysis of the segments 102 a and 102 b, and/or theinformation 122 is provided to the Measurement ERP 118 to improve itsperformance.

With computer games there are often many opportunities to substantiallyinstantaneously alter how the game proceeds (e.g., difficulty ofmissions, appearance of characters). Therefore, being able to measurethe affective response of the user 114 to content generated by thecomputer game 120 can be beneficial since it can help guide the systemtowards alterations that can improve the user's playing experience. Forexample, if the user's affective response indicates that the user islosing interest and is not excited by playing at a certain level ofdifficulty, the game can increase the difficulty and add surprisingeffects and/or plot twists in order to make the game more exciting forthe user.

In one example, the computer game 120 provides the content ERA 104 withcontext information regarding an event in the game that is related tothe segments of content 102 a and 102 b. For instance, the computer gamemay provide the content ERA with information indicative of the fact thatthe user 114 has been playing the same level for a long time, which mayindicate that the user might be getting bored and/or frustrated with thegame. In another example, the computer game 120 provides the content ERA104 with information regarding the status of events in the game notrepresented in the analyzed content (such as the state of othercharacters in the game, and/or how the user 114 stands with respect tocompleting a mission). This information can provide context and assistthe content ERA in determining the user's expected emotional response tothe content, and thus improve the accuracy of selection of appropriatemodes of operation for measuring the affective response to differentsegments of content. In yet another example, the computer game 120includes a character whose actions are controlled at least partially bythe user, and the information provided by the game to the content ERA104 is related to the actions of the character and/or to the fate ofcharacter in the game.

In one embodiment, the functionality of the content ERA 104 is includedin the interactive computer game 120. In this embodiment, the computergame provides indications to the controller 108 that selects a mode ofoperation for the device 112. In another embodiment, the functionalityof both the content ERA 104 and the controller 108 are included in theinteractive computer game 120. In this case, the hardware of thecomputer game may control the device 112.

In one embodiment, the system 100 includes a measurement EmotionalResponse Predictor (measurement ERP) configured to predict the user'semotional response from data comprising measurements 113, which may bemeasurements of affective response. Optionally, the measurements 113 areprocessed prior to being sent to the measurement ERP 118 for prediction.For example, the measurements 113 may undergo filtering and/or featureextraction. Additionally, the measurement ERP 118 may be provided withinformation generated by the content ERA's 104 analysis of a segment ofcontent, such as the indication 106 a or 106 b. The measurement ERP 118may utilize this information for generating its prediction of the user'semotional response.

In one embodiment, the system 100 further includes a module for trainingan emotional response predictor. The measurements 113 of the user'saffective response to a segment of content may be forwarded to themodule to be used for creating training samples for training thepredictor. Additionally, the measurements 113 may be processed prior tobeing sent to the module to be used for training; for instance, themeasurements may undergo filtering and/or feature extraction in order toproduce feature values that are included in one or more training samplesutilized by the module. Optionally, the module is provided withindications generated by the content ERA 104, and/or other informationgenerated by the content ERA, which are utilized to create a label forone or more training samples.

FIG. 13 illustrates one embodiment of an apparatus configured to powerconsumed measuring affective response. The apparatus includes at least areceiver 143, a processor 145, and a controller 147.

The receiver 143 is configured to receive a first indication 142 a and asecond indication 142 b from a content ERA 141. Optionally, the contentERA 141 produces the indications 142 a and 142 b by analyzing segmentsof content 102 a and 102 b.

In one embodiment, the first indication 142 a indicates that a firstvalue, which is a function of emotional response the first segment 102a, does not reach a first predetermined threshold. Additionally, thesecond indication 142 b indicates that a second value, which is afunction of emotional response to the second segment 102 b, does reach asecond predetermined threshold. Optionally, the first and/or secondvalues are derived by utilizing a content ERP that predicts emotionalresponse to the first and/or second segments. Optionally, the first andsecond values may represent confidence levels in predictions ofemotional response to the first and second segments, types of emotionalresponse to the first and second segments, magnitude of emotionalresponses to the first and second segments, and/or benefits frommeasuring affective response to the first and second segments.

The processor 145 is configured to select, based on indications 142 aand 142 b received by the receiver 143, a first mode of operation 146 aand a second mode of operation 146 b, for the device 112 for measuringaffective responses of the user to the first segment 102 a and thesecond segment 102 b, respectively.

In one embodiment, the controller 147 is configured to manage powerconsumption of the device 112 according to the mode of operationselected by the processor 145. Optionally, managing the powerconsumption of the device is achieved by configuring the device 112 tooperate in a certain mode of operation, such as the first mode 146 a orthe second mode 146 b. Optionally, the first mode 146 a and the secondmode 146 b are selected such that per unit of measurement time, thepower consumed by the device 112 measuring the affective response of theuser 114 to the first segment 102 a is significantly lower than thepower consumed by the device 112 measuring the affective response of theuser 114 to the second segment 102 b. Thus, configuring the device 112to operate in the first and/or second modes of operation to measureaffective response of the user to at least some of the segments mayresult in a reduction of power consumed by the device 112.

In one embodiment, a training module 139 configured to train anemotional response predictor such as a content ERP or a measurement ERPreceives measurements 113 of the user 114 taken by the device 112.Optionally, the training module 139 receives the indication 142 b andutilizes to generate a training sample comprising measurements of theaffective response of the user 114 to the second segment 102 b and alabel derived from the indication 142 a representing the expectedemotional response to the second segment 102 b.

In one embodiment, a system configured to reduce power consumptionaccording to analysis of content, comprising: a content emotionalresponse analyzer (content ERA) configured to: receive a first segmentof content, analyze the first segment, and output a first indicationthat a first value related to a predicted emotional response to thefirst segment does not reach a first predetermined threshold; and acontroller configured to select, based on the first indication, a firstmode of operation for a device for measuring affective response of auser to the first segment; the content ERA is further configured to:receive a second segment of content, analyze the second segment, andoutput a second indication that a second value related to a predictedemotional response to the second segment reaches a second predeterminedthreshold; the controller is further configured to select, based on thesecond indication, a second mode of operation for the device formeasuring affective response of the user to the second segment; whereinthe device is configured to consume significantly less power whileoperating in the first mode of operation compared to the power it isconfigured to consume while operating in the second mode of operation.Optionally, the first and second predetermined thresholds representfirst and second confidence levels in predictions of emotional response,respectively; and wherein the first and second values representconfidence in predictions of emotional response to the first and secondsegments, respectively; whereby the first mode of operation is selectedwhen the first indication indicates that confidence in a prediction ofemotional response to the first segment does not reach the firstconfidence level, and the second mode of operation is selected when thesecond indication indicates that confidence in a prediction of emotionalresponse to the second segment does reach the second confidence level.Optionally, the first and second predetermined thresholds representfirst and second magnitudes of change in affective response,respectively; and wherein the first and second values representpredicted changes in affective response due to exposure to the first andsecond segments, respectively; whereby the first mode of operation isselected when the first indication indicates that a predicted change inaffective response due to exposure to the first segment does not reachthe first magnitude, and the second mode of operation is selected whenthe second indication indicates that a predicted change in affectiveresponse due to exposure to the second segment does reach the secondmagnitude. Optionally, the first and second predetermined thresholdsrepresent first and second benefit levels, respectively; and wherein thefirst and second values represent predicted benefits to measuringaffective response to the first and second segments, respectively;whereby the first mode of operation is selected when the firstindication indicates that a predicted benefit of measuring affectiveresponse to the first segment does not reach the first benefit level,and the second mode of operation is selected when the second indicationindicates that a benefit of measuring affective response to the secondsegment does reach the second benefit level. Optionally, the first andsecond segments comprise visual content, and being exposed to the firstand second segments is achieved by viewing the first and secondsegments. Optionally, the first and second segments comprise datarepresenting text, and the content ERA utilizes semantic analysis of thetext to produce the first and second indications. Optionally, the deviceis battery powered and is one or more of the following: a physiologicalsensor, and an image capturing device. Optionally, the system furthercomprises a measurement emotional response predictor (measurement ERP)configured to predict emotional response of the user from datacomprising the measurements of the device. Optionally, the first andsecond segments are received from a computer game that further providesinformation about expected emotional response to the first and secondsegments.

In one embodiment, a method for reducing power consumed measuringaffective response, comprising: receiving a first indication derivedfrom analysis of a first segment of content; determining that the firstindication indicates that a first value related to a prediction ofemotional response to the first segment does not reach a firstpredetermined threshold, and configuring a device to operate in a firstmode of operation to measure affective response of a user to the firstsegment; receiving a second indication derived from analysis of a secondsegment of content; and determining that the second indication indicatesthat a second value related to a prediction of emotional response to thesecond segment reaches a second predetermined, and configuring thedevice to operate in a second mode of operation to measure affectiveresponse of the user to the second segment; wherein power consumed bythe device, per unit of measurement time, for measuring the affectiveresponse of the user to the first segment, is significantly lower thanpower consumed by the device for measuring the affective response of theuser to the second segment. Optionally, the first and second segmentscomprise data representing text, and the analysis of the first andsecond segments utilizes semantic analysis of the text to produce thefirst and second indications. Optionally, power consumed measuring theuser with the device while operating in the first mode of operation issubstantially zero; whereby power consumption of substantially zero isachieved by substantially not measuring the user with the device.Optionally, the method further comprises forwarding, to a module thattrains an emotional response model, a description of expected emotionalresponse to the second segment and data obtained from the measuring theaffective response of the user to the second segment. Optionally, thefirst and second indications indicate confidence in predictions ofemotional response to the first and second segments, respectively;whereby based on the first indication, it is determined that confidencein the prediction of the emotional response to the first segment doesnot reach the first predetermined threshold, and based on the secondindication, it is determined that confidence in the prediction of theemotional response to the second segment does reach the secondpredetermined threshold. Optionally, the first and second indicationsindicate expected magnitudes of change in affective response due toexposure to the first and second segments, respectively; whereby basedon the first indication, it is determined that expected magnitude ofchange in affective response due to exposure to the first segment doesnot reach the first predetermined threshold, and based on the secondindication, it is determined that expected magnitude of change toaffective response due to exposure to the second segment does reach thesecond predetermined threshold. Optionally, the first and secondindications indicate how beneficial it is to measure the affectiveresponse to the first and second segments, respectively; whereby basedon the first indication, it is determined that benefit from measuringthe affective response to the first segment does not reach the firstpredetermined threshold, and based on the second indication it isdetermined that benefit from measuring affective response to the secondsegment does reach the second predetermined threshold. Optionally, thedevice is battery powered, and measuring the affective response of theuser comprises utilizing the device to take physiological measurementsof the user. Optionally, the device is part of a battery powered imagecapturing device, and measuring the affective response of the usercomprises utilizing the device to capture an image of the user.Optionally, the second segment is received from an interactive computergame that further provides information about expected emotional responseto the second segment.

In one embodiment, an apparatus configured to reduce power consumedmeasuring affective response, comprising: a receiver configured toreceive first and second indications generated from analysis of firstand second segments of content, respectively; wherein the firstindication indicates that a first value, which is a function ofemotional response the first segment, does not reach a firstpredetermined threshold; and wherein the second indication indicatesthat a second value, which is a function of emotional response to thesecond segment, reaches a second predetermined threshold; a processorconfigured to select, based on the first and second indications, firstand second modes of operation for device for measuring affectiveresponses of a user to the first and second segments, respectively; anda controller configured to manage power consumption of the deviceaccording to a mode of operation selected by the processor; wherein, perunit of measurement time, the power consumed by the device for measuringthe affective response of the user to the first segment is significantlylower than the power consumed by the device for measuring the affectiveresponse of the user to the second segment.

FIG. 16 illustrates one embodiment of a system 150 configured to reducecomputational load involved in processing affective responsemeasurements. The system includes at least at least a content EmotionalResponse Analyzer (content ERA) 104, a controller 154, and a processor156.

In one embodiment, the content ERA 104 may outputs the first indication106 a and the second indication 106 b, which may be conveyed to thecontroller 154. The controller 154 is configured to select a processinglevel for the processor 156, based on the indications. Optionally, theindications 106 a and 106 b may be received separately by the controller154 (e.g., each indication is communicated in a separate communication),or may be received jointly by it (e.g., both indications arecommunicated to the controller 154 in the same communication). In oneexample, based on the indication 106 a, the controller 154 is configuredto select for the processor 156 a first processing level 155 a, andbased on the second indication 106 b, the controller 154 is configuredto select for the processor 156 a second processing level 155 b.Optionally, at least during some of the time it operates at the firstprocessing level 155 a and/or the second processing level 155 b, theprocessor 156 processes affective response measurements taken with adevice 148. Optionally, at least some of the affective responsemeasurements taken by the device 148 are measurements of the affectiveresponse of the user 114 to the segment of contents 102 a and/or 102 b.

In one embodiment, the indication 106 a and/or the indication 106 b maybe interpreted as operating instructions for another module in thesystem, such as the controller 154 and/or the processor 156. Forexample, the indications 106 a and/or 106 b may be interpreted asinstructions to the processor 156 to run a certain procedure and/or asan instruction to the controller 154 to select a specific processinglevel for the processor 156. In another example, the indications 106 aand/or 106 b may indicate how long the processor 156 should process dataand/or specify a period of time (e.g., by providing start and end times)in which the processor 156 should process data.

In some embodiments, a device, such as the device 148, is used tomeasure affective response of the user 114. Optionally, the deviceincludes a sensor and/or is a sensor. Optionally, the sensor may be aphysiological sensor (e.g., a sensor that measures heart rate, galvanicskin response, and/or brainwave activity), and/or a sensor that measuresthe user's behavior (e.g., a camera, and/or a motion detector).Optionally, the device may include additional components to the sensor,such as a memory, a processor, a battery, a transmitter, and/or areceiver. Optionally, the device may be coupled to a user. Herein aphrase like “a device coupled to a user” refers to a device that isattached to the user (e.g., on clothing, a bracelet, headgear), incontact with the user's body (e.g., a sticker on a user's skin), and/orimplanted in the user's body (e.g., an implanted heart-rate sensor orimplanted EEG electrodes).

In some embodiments, the device 148 may output measurements of theaffective response of the user 114. Optionally, the measurements areforwarded to the processor 156 and undergo processing, such asfiltering, normalization, signal processing, feature extraction,encryption, and/or compression. Additionally, affective responsemeasurements processed by the processor 156 may be forwarded to othermodules of the system such as the measurement Emotional ResponsePredictor (measurement ERP) 158. Optionally, some of the measurementsfrom the device 148 are forwarded to the measurement ERP 158, and mayoptionally include raw measurement values (i.e., the values undergo verylittle transformation prior to their output).

In some embodiments, the controller 154 is configured to utilize theindications 106 a and/or 106 b to select a processing level for theprocessor 206 from a set comprising a first processing level 155 a and asecond processing level 155 b. The processing level may have presetoperation parameters (such as procedures to run, duration of processing,and/or volume of data to process). Alternatively or additionally, theprocessing level may include certain operation parameters that arecomputed by the controller using the indications 106 a and/or 106 b,and/or other data that is available at the time of computation.

In one example, the controller chooses a processing level for theprocessor from two processing level: a standby processing level and anormal processing level. In this example, as is the case with manyprocessors, when a processor operates at the standby processing level,it substantially doesn't process data, but monitors communicationchannels to receive instructions (such as a wake up signal). Whenoperating at the standby processing level, the processor utilizessignificantly fewer computation cycles compared to when it operates atthe normal processing level. For example, a processor in a standby modemy run at a lower clock frequency or utilize fewer computational cores.

In another example, the affective response of the user 114 is recordedwith a camera. The Indications generated by a content ERA and/or asemantic analyzer may include values that is indicative of a predictedinterest level of a user in segments of content. According to thepredicted interest level, processing levels with different parametersmay be selected for processing the video images of the user 114. Forexample, if an indication indicates that the predicted interest level islow, the controller may select the first processing level in which theprocessor may perform basic filtering and/or low-level featureextraction from the video; such rudimentary processing may be requiredfor identifying the genre of video from low level features. However, ifan indication indicates the predicted interest level is high, a secondprocessing level may be selected in which the processor performsadvanced compute-intensive procedures such as object, face, and/orgesture recognition in order to identify things such as the objects,people, and/or emotions expressed in the recorded video. Thus, in thisexample, the computational load of the processor operating at the firstprocessing level is significantly lower than the computational load ofthe processor operating at the second processing level.

In one embodiment, as part of its analysis of the segment of content,the content ERA 104 and/or the semantic analyzer 173 predicts a valuedescribing the expected emotional response of a user to the segment ofcontent. An indication generated by the content ERA 104 and/or thesemantic analyzer 173 may include information regarding the expectedemotional response and is conveyed via indications that may beindicative of whether the expected emotional response reaches apredetermined threshold. Based on the indications, different processinglevels may be selected. The indications may indicate on a scale of 1 to10 (10 being the extremely frightened), how frightened a user is likelyto get from viewing a segment of content that is a video clip. Based ona received indication, a controller selects a certain processing levelfor processing of affective response measurements of the user 114 (e.g.,brainwaves acquired by an EEG sensor). In order to save computationalresources, the system may elect to process the affective responsemeasurements of the user 114 mainly when the emotional response isexpected to reach a predetermined threshold, such as 5 on the scale of 1to 10. In this case, processing the measurements may help pinpoint whatelements in the content led to the user's affective response. However,if the predicted value is too low, it is not likely that the clip isgoing to scare the user 114 very much, so the system chooses not towaste computational resources on confirming that. Thus, if the expectedemotional response is below the threshold, the controller may select afirst processing level, which may be a standby processing level, for theprocessor. However, if the expected emotional response reaches thepredetermined threshold, the controller may select the second processinglevel, which may be a regular processing level, for the processor. Inthe normal processing level, the processor utilizes significantly morecomputational cycles than it does when it operates at the standbyprocessing level. Consequently, the computational load of the processoris significantly lower when operating at the first processing level,compared to when it operates at the second processing level.

In one embodiment, a processing level may be implicitly or explicitlyselected by the controller. For example, an implicit selection of theprocessing level may occur when the controller does not receive anindication from the content ERA (e.g., when the indication has not beenchanged since the last signaling, and/or when the indication is adefault indication); an explicit selection of the processing level mayoccur when an indication received by the controller explicitly mentionsthe processing level that is to be set by the controller.

In some embodiments, the processor 156 is configured to processmeasurements of affective response of the user 114, while operating at acertain processing level. The certain processing level may be selectedfrom a set that includes at least the first processing level 155 a andthe second processing level 155 b. The computational load of theprocessor may be expressed in terms of the number of computationalcycles the processor utilizes to process a volume unit of data.

In one embodiment, based on the first indication 106 a, the controller154 selects the first processing level 155 a, with a low computationalload, at which the processor 156 should operate while processingmeasurements of affective response of the user 114 to the first segment102 a. In addition, based on the second indication 106 b, the controller154 selects the second processing level 155 b, with a highercomputational load, at which the processor 156 should operate whileprocessing measurements of affective response of the user 114 to thesecond segment 102 b. Consequently, per volume unit of measurement data,the number of computation cycles utilized for processing measurements ofaffective response of a user to the first segment, is significantlylower than number of computation cycles utilized for processingmeasurements of affective response of the user to the second segment.

It is to be noted that a term such as “significantly lower”, whenreferring to the number of computation cycles, refers to a differencegreater than a factor of two between the lower and higher numbers ofcomputation cycles being compared. To illustrate this point, consider aprocessor capable of operating at two processing levels: A and B. Whenit is stated that the processor utilizes a significantly lower number ofcomputation cycles while operating at processing level A compared to thenumber of computation cycles it utilizes while operating at processinglevel B, it is meant that the number of computation cycles utilized bythe processor for processing data, per volume unit of data, whileoperating at processing level A, is less than half of the number ofcomputation cycles utilized by the processor, per volume unit ofmeasurement data, when it operates at processing level B.

It is to be noted that a phrase like “computation cycles per volume unitof measurement data” refers to the number of computational cycles usedto process a fixed volume of affective response measurement data (suchas a byte, a kilobyte, a megabyte, or a gigabyte). The valuerepresenting the number of computation cycles utilized by a processorper volume unit of data is computed by dividing the total number ofcomputation cycles utilized to process data by the volume of the data(e.g., expressed in bytes, kilobytes, or megabytes). This value servesas a normalized value that enables the comparison of computational loadsof a processor operating at different processing levels, and/ordifferent processors operating at the same or different processinglevels.

A computation cycle, as used herein, refers to a pulse of a CPU's clockin which a CPU core typically performs a single simple computationoperation (such as adding two numbers, or storing a value in aregister). Modern CPUs with clock frequencies that are measured inGigahertz (GHz) perform billions of computation cycles per second, andconsequently are capable of performing billions of computationoperations each second. In typical computational applications, theamount of computational work that goes into processing data isproportional to the volume of the data, i.e., the amount of processingtime and/or computation cycles utilized to process data is a function ofthe volume of the data. This makes it possible to refer to the work loadinvolved in processing data by way of the number of computation cycles aprocessor typically utilizes to process a certain volume unit of data.

In one embodiment, two processing levels may be considered differentprocessing levels, and be referred to as “a first processing level” and“a second processing level” in the claims, if the difference, per volumeunit of data, between the number of computational cycles utilized by aprocessor processing measurements at the first processing level and thenumber of computational cycles utilized by a processor processingmeasurements at the second processing level, is significant. If thedifference is not significant, then the two processing levels are not “afirst processing level” and “a second processing level” as used in theclaims. In one example, a significant difference refers to a differenceof at least 100% in the number of computation cycles, e.g., theprocessor 156 processing 1 kilobyte of measurement data at the secondprocessing level 155 b utilizes at least double the number ofcomputation cycles as the processor 156 utilizes to process the same 1kilobyte of data at the first processing level 155 a. In anotherexample, a significant difference refers to a difference of at least1000% in the number of computation cycles utilized by the first andsecond processing levels. In yet another example, if when processingmeasurement data at the first processing level 155 a, the processor 156utilizes essentially zero computation cycles (e.g., it doesn't processthe measurement data), and at the second processing level 155 b, theprocessor 156 does not utilize essentially zero computation cycles(e.g., it performs some processing of the data), then the differencebetween the processing levels 155 a and 155 b is significant.

There may be various hardware configurations that involve the contentERA 104, the controller 154, and/or the processor 156. In one example,the controller 154 may be implemented, at least in part, as hardwarethat is part of the processor 156. In another example, the controller154 may be implemented, at least in part, as software that runs on theprocessor 156. In another example, the controller 154 may be implementedas part of the content ERA 104. In still another example, the controller154, the processor 156, and the content ERA 104 are combined in acertain device. Optionally, the certain device is battery powered (e.g.,a smart phone). Optionally, the device 148 is also part of the certaindevice and/or draws power from the certain device's power supply.

In one embodiment, the processor 156 is embedded in a certain batterypowered device, and the measurements of affective response of the userto the second segment include physiological measurements of the usertaken with the device 148 that includes a physiological sensor.Additionally or alternatively, the measurements of affective response ofthe user to the second segment comprise images of the user taken with animage capturing device.

In one embodiment, a segment of content may include visual content, suchas images and/or video. A content delivery module may be used to exposethe segments to the user 114. Optionally, the device 148 has at leastone component that is not shared with the content delivery module.Additionally or alternatively, the device 148 may include at least onecomponent that is shared with the content delivery module. Additionallyor alternatively, the device 148 may be a battery powered sensor, andthe content delivery module and the battery powered sensor are realizedin the same device.

In one embodiment, the controller 154 and/or the content ERA 104 areimplemented at least in part, as software running on a remote processor,such as a cloud-based server. For example, a cloud-based service can beused to analyze content that is to be presented to the user 114, such asvideo to be streamed to the user's display. According to indications ofthe content ERA's analysis, the controller 154, running on thecloud-based service, may send different operating instructions to theprocessor 156, such as specifying a processing level for the processor156 at to operate while processing affective response measurements ofthe user.

In one embodiment, the functionality of the content ERA 104 is includedin the interactive computer game 120 illustrated in FIG. 15. In thisembodiment, the computer game provides indications to the controller 154that selects a processing level for the processor 156. In anotherembodiment, the functionality of both the content ERA 104 and thecontroller 154 are included in the interactive computer game 120. Inthis case, the hardware of the computer game may control the device 148.

In one embodiment, the system 150 includes a measurement EmotionalResponse Predictor (measurement ERP) 158 configured to predict theuser's emotional response from data comprising measurements of affectiveresponse. Optionally, the measurements are processed by the processor156 prior to being sent to the measurement ERP 158 for prediction. Forexample, the measurements may undergo filtering and/or featureextraction. Additionally, the measurement ERP 158 may be provided withinformation generated by the content ERA's 104 analysis of segment ofcontent, such as the indications 106 a and/or 106 b. The measurement ERP158 may utilize this information for generating its prediction of theuser's emotional response.

In one embodiment, the system 150 further includes a module for trainingan emotional response predictor. The measurements of the user'saffective response to segments of content may be forwarded to the moduleto be used for creating training samples for training the predictor.Additionally, the measurements may be processed by the processor 156prior to being sent to the module to be used for training; for instance,the measurements may undergo filtering and/or feature extraction inorder to produce feature values that are included in one or moretraining samples utilized by the module. Optionally, the module isprovided with the indications 106 a and/or 106 b, and/or otherinformation generated by the content ERA 104, which are utilized tocreate a label for one or more training samples.

FIG. 17 illustrates one embodiment of a method for reducingcomputational load of processing measurements of affective response tocontent. The method includes the following steps:

In step 160, receiving a first indication derived from analysis of afirst segment of content.

In step 161, determining that the first indication indicates that afirst value related to a prediction of emotional response to the firstsegment does not reach a first predetermined threshold.

In step 162, configuring a processor, based on the first indication, tooperate at a first processing level to process measurements of affectiveresponse of the user to the first segment.

In step 163, receiving a second indication derived from analysis of asecond segment of content.

In step 164, determining that the second indication indicates that asecond value related to a prediction of emotional response to the secondsegment does reach a second predetermined.

And in step 165, configuring the processor, based on the secondindication, to operate at a second processing level to processmeasurements of affective response of the user to the second segment.

In one embodiment, per volume unit of measurement data, number ofcomputation cycles utilized by the processor to process, at the firstprocessing level, measurements of the affective response of the user tothe first segment, is significantly lower than the number of computationcycles utilized by the processor to process, at the second processinglevel, measurements of the affective response of the user to the secondsegment.

In one embodiment, a processor is configured to utilize significantlyless computation cycles per volume unit of data (e.g., per kilobyte ofmeasurement data) to process data when operating in a first processinglevel, compared to the number of computation cycles utilized per volumeunit of data when operating at a second processing level. Additionally,the processor operating at the second processing level may requiresignificantly more time and/or consume significantly more energycompared to when the processor operates at the second processing level.Optionally, the phrase “significantly more” when referring to processingcycles, means at least double, and similarly, the phrase “significantlyless” means at most half. Thus for example, by stating that “the numberof computation cycles utilized for processing the measurements ofaffective response of a user to the first segment, is significantlylower than the number of computation cycles utilized for processingmeasurements of affective response of the user to the second segment”,it is meant that to process the same volume of data, the processorutilizes at most half the number of computation when operating at thefirst processing level, compared to the number of computation it cyclesit utilizes when operating at the second processing level.

In one embodiment, a processor configured to operate at a firstprocessing level that utilizes significantly less computation cycles forprocessing measurements of affective response of a user to the firstsegment, compared to the number of computational cycles utilized by theprocessor for processing measurements of affective response of a user tothe second segment. Optionally, the processor utilizes substantiallyzero computation cycles to process data when operating at the firstprocessing level. For example, the processor is placed in a standbymode, does not receive and/or access data that includes the availablemeasurements of affective response of a user to the first segment,and/or deletes data involving the measurements of affective response ofa user to the first segment.

In one embodiment, the first and second segments are received from aninteractive computer game module. The module provides informationregarding an expected emotional response to the second segment, which isused in order to generate the second indication. Additionally, thecomputer game may also provide information regarding an expectedemotional response to the first segment, which is used to generate thefirst indication.

In one embodiment, the first and second segments include datarepresenting text. The analysis of the first and second segmentsutilizes semantic analysis of the data representing text to produce theindications that indicate whether or not the predetermined threshold isreached. Additionally, the semantic analysis may be used to generatefeature values that describe the meaning of the text of the first andsecond segments. The feature values may be used by the content ERA tomake predictions related to the emotional response to the content and/orthe benefit of determining the emotional response to the first and/orsecond segments.

In one embodiment, a system configured to reduce computational loadinvolved in processing measurements of affective response of a user tocontent, comprising: a content emotional response analyzer (content ERA)configured to: receive a first segment of content, analyze the firstsegment, and output a first indication indicating that a first valuerelated to an emotional response to the first segment does not reach afirst predetermined threshold; and a controller configured to select,based on the first indication, a first processing level for a processorto process measurements of affective response of the user to the firstsegment; the content ERA is further configured to: receive a secondsegment of content, analyze the second segment, and output a secondindication indicating that a second value related to a prediction ofemotional response to the second segment reaches a second predeterminedthreshold; the controller is further configured to select, based on thesecond indication, a second processing level for the processor toprocess measurements of affective response of the user to the secondsegment; wherein, per volume unit of measurement data, number ofcomputation cycles utilized by the processor to process, at the firstprocessing level, measurements of the affective response of the user tothe first segment, is significantly lower than number of computationcycles utilized by the processor to process, at the second processinglevel, measurements of the affective response of the user to the secondsegment. Optionally, the first and second indications indicateconfidence in predictions of emotional response to the first and secondsegments, respectively; whereby based on the first indication, it isdetermined that confidence in the prediction of the emotional responseto the first segment does not reach the first predetermined threshold,and based on the second indication, it is determined that confidence inthe prediction of the emotional response to the second segment doesreach the second predetermined threshold. Optionally, the first andsecond indications indicate expected magnitudes of change in affectiveresponse due to exposure to the first and second segments, respectively;whereby based on the first indication, it is determined that expectedmagnitude of change in affective response due to exposure to the firstsegment does not reach the first predetermined threshold, and based onthe second indication, it is determined that expected magnitude ofchange to affective response due to exposure to the second segment doesreach the second predetermined threshold. Optionally, the first andsecond indications indicate how beneficial it is to measure theaffective response to the first and second segments, respectively;whereby based on the first indication, it is determined that benefitfrom measuring the affective response to the first segment does notreach the first predetermine threshold, and based on the secondindication it is determined that benefit from measuring affectiveresponse to the second segment does reach the second predeterminedthreshold. Optionally, the first and second segments comprise visualcontent, and being exposed to the first and second segments is achievedby viewing the first and second segments. Optionally, the first andsecond segments comprise data representing text, and the content ERAutilizes semantic analysis of the text to produce the first and secondindications. Optionally, measurements of the affective response of theuser to the first and second segments are acquired by a sensor; whereinthe sensor is battery powered and is one or more of the following: aphysiological sensor, and an image capturing device. Optionally, thenumber of computation cycles utilized to process affective responsewhile operating at the first processing level is substantially zero;whereby utilizing substantially zero computation cycles is achieved bysubstantially not processing measurement of affective response.Optionally, comprising a measurement emotional response predictor(measurement ERP) configured to predict emotional response of the userfrom data comprising measurements of a sensor. Optionally, the first andsecond segments are received from a computer game that further providesinformation about expected emotional response to the first and secondsegments.

In one embodiment, a method for reducing computational load ofprocessing measurements of affective response of a user to content,comprising: receiving a first indication derived from analysis of afirst segment of content; determining that the first indicationindicates that a first value related to a prediction of emotionalresponse to the first segment does not reach a first predeterminedthreshold, and configuring a processor to operate at a first processinglevel to process measurements of affective response of the user to thefirst segment; receiving a second indication derived from analysis of asecond segment of content; and determining that the second indicationindicates that a second value related to a prediction of emotionalresponse to the second segment reaches a second predetermined threshold,and configuring the processor to operate at a second processing level toprocess measurements of affective response of the user to the secondsegment; wherein, per volume unit of measurement data, number ofcomputation cycles utilized by the processor to process, at the firstprocessing level, measurements of the affective response of the user tothe first segment, is significantly lower than number of computationcycles utilized by the processor to process, at the second processinglevel, measurements of the affective response of the user to the secondsegment. Optionally, the first and second predetermined thresholdsrepresent first and second confidence levels in predictions of emotionalresponse, respectively; and wherein the first and second valuesrepresent confidence in predictions of emotional response to the firstand second segments, respectively; whereby the processor is configuredto operate at the first processing level to process measurements ofaffective response of the user to the first segment when the firstindication indicates that confidence in a prediction of emotionalresponse to the first segment does not reach the first confidence level,and the processor is configured to operate at the second processinglevel to process measurements of affective response of the user to thesecond segment when the second indication indicates that confidence in aprediction of emotional response to the second segment does reach thesecond confidence level. Optionally, the first and second predeterminedthresholds represent first and second magnitudes of change in affectiveresponse, respectively; and wherein the first and second valuesrepresent predicted changes in affective response due to exposure to thefirst and second segments, respectively; whereby the processor isconfigured to operate at the first processing level to processmeasurements of affective response of the user to the first segment whenthe first indication indicates that a predicted change in affectiveresponse due to exposure to the first segment does not reach the firstmagnitude, and the processor is configured to operate at the secondprocessing level to process measurements of affective response of theuser to the second segment when the second indication indicates that apredicted change in affective response due to exposure to the secondsegment does reach the second magnitude. Optionally, the first andsecond predetermined thresholds represent first and second benefitlevels, respectively; and wherein the first and second values representpredicted benefits to measuring affective response to the first andsecond segments, respectively; whereby the processor is configured tooperate at the first processing level to process measurements ofaffective response of the user to the first segment when the firstindication indicates that a benefit of measuring affective response tothe first segment does not reach the first benefit level, and theprocessor is configured to operate at the second processing level toprocess measurements of affective response of the user to the secondsegment when the second indication indicates that a benefit of measuringaffective response to the second segment does reach the second benefitlevel. Optionally, the first and second segments comprise datarepresenting text, and the analysis of the first and second segmentsutilizes semantic analysis of the text to determine whether thepredetermined threshold is reached. Optionally, the number ofcomputation cycles utilized to process the measurements at the firstprocessing level is substantially zero; whereby utilizing substantiallyzero computation cycles is achieved by substantially not processingmeasurements of affective response. Optionally, the method furthercomprises forwarding, to a module that trains an emotional responsemodel, a description of expected emotional response to the secondsegment and data obtained from processing the measurements of theaffective response of the user to the second segment. Optionally, themethod further comprises forwarding data obtained from processing themeasurements of the affective response of the user to the second segmentto a module that predicts emotional response. Optionally, the secondsegment is received from an interactive computer game that furtherprovides information about expected emotional response to the secondsegment.

In one embodiment, a system configured to reduce computational loadinvolved in predicting emotional response to content, comprising: acontent emotional response analyzer (content ERA) configured to: receivea first segment of content, analyze the first segment, and output afirst indication that a first value related to a predicted emotionalresponse to the first segment does not reach a first predeterminedthreshold; the content ERA is further configured to: receive a secondsegment of content, analyze the second segment, and output a secondindication that a second value related to a predicted emotional responseto the second segment reaches a second predetermined threshold; acontroller configured to select, based on the first indication, a firstprocessing level for a processor to process measurements of affectiveresponse of a user to the first segment; the controller is furtherconfigured to select, based on the second indication, a secondprocessing level for the processor to process measurements of affectiveresponse of the user to the second segment; wherein, per volume unit ofmeasurement data, number of computation cycles utilized by the processorto process, at the first processing level, measurements of the affectiveresponse of the user to the first segment, is significantly lower thannumber of computation cycles utilized by the processor to process, atthe second processing level, measurements of the affective response ofthe user to the second segment; and a measurement emotional responsepredictor configured to predict emotional response of the user to thesecond segment of content from data comprising the measurements of theaffective response of the user to the second segment, which wereprocessed by the processor.

FIG. 18 illustrates one embodiment of a system 170 configured to measureaffective response at varying rates that are set according to semanticanalysis of text. The system includes at least a semantic analyzer 173and a controller 175.

In one embodiment, the semantic analyzer 173 is configured to: (i)receive a first segment 172 a of content and a second segment 172 b ofcontent, (ii) analyze the segments 172 a and 172 b utilizing semanticanalysis, and (iii) output a first indication 174 a and a secondindication 174 b. The first segment 172 a and second segment 172 binclude data representing text. The first indication 174 a indicatesthat a first value related to a prediction of emotional response to thefirst segment 172 a does not reach a first predetermined threshold. Thesecond indication 174 b indicates that a second value related to aprediction of emotional response to the second segment 172 b does reacha second predetermined threshold.

In one embodiment, the segment of content might undergo processingbefore being received by the semantic analyzer 173 and/or before thesemantic analyzer 173 analyzes the segment of content. Thus, dataderived from the segment of content, which gets analyzed by the semanticanalyzer 173, may not be the same data presented to the user 114. Forexample, a segment of content that includes video may undergo featureextraction prior to being sent to the semantic analyzer 173 (e.g., inorder to identify object, people, and/or the genre). In this example,the exposure of the user 114 to the segment of content involves viewinga video clip. However, the semantic analyzer 173 receives related data,such as feature values corresponding to objects, people, and/or thegenre of the video, and performs its analysis on that data. In anotherexample, a segment of content is converted to features representingwords (e.g., using a “bag of words” model).

Being able to determine the meaning of a segment of content can assistalgorithms to predict an emotional response to the segment of content.In some embodiments, the semantic analyzer 173 utilizes semanticanalysis methods to analyze the segments of content 172 a and 172 b.Optionally, the semantic analysis methods are used to generate featurevalues that describe some aspects of the meaning of the segments 172 aand 172 b. Additionally, the feature values may be used by algorithmsthat predict an emotional response to the segment 172 a and 172 b.

In one example, the semantic analyzer may utilize methods such as LatentSemantic Analysis (LSA) or Latent semantic indexing of text in order toassociate a segment of content with concepts and/or categoriescorresponding to its meaning. In another example, the semantic analyzerutilizes a lexicon that associates words and/or phrases with coreemotions. The information regarding which emotions are expressed in asegment of content can be helpful in determining the meaning and/orsentiment expressed in the segment of content. In yet another example,the semantic analyzer utilizes a predictor of emotions expressed intext. Information regarding the emotion expressed in the segment ofcontent can help to predict a user's emotional response to the content(e.g., a user might enjoy segment that expresses affection, but dislikea segment that expresses confrontation).

In one embodiment, the semantic analyzer evaluates the segment a shortperiod before the user 114 is exposed to the segment. For example, thesemantic analyzer evaluates the second segment 172 b less than oneminute before the user 114 is exposed to the second segment 172 b.Optionally, the segment 172 b is generated shortly before it isanalyzed. For example, the segment 172 b is generated by a computergame, and the content of the segment 172 b depends at least in part onactions taken by the user.

In one embodiment, if the evaluation of the semantic analyzer 173determines that a predefined threshold related to an emotional responseis not reached, this fact may be communicated to a module that generatedthe segment of content. The module that generated the segment of contentmay modify the segment of content so that the predefined threshold maybe reached in a future evaluation performed by the semantic analyzer onthe modified segment of content.

In one embodiment, the semantic analyzer utilizes a content ERP(Emotional Response Predictor) that predicts an emotional response to asegment of content. Optionally, the content ERP utilizes feature valuesderived using semantic analysis methods.

In one embodiment, the semantic analyzer 173 may generate the firstindication 174 a and the second indication 174 b, which are conveyed tothe controller 175. The controller 175 is configured to select ameasuring rate for a device 177, based on a received indication. Forexample, based on the indication 174 a, the controller 175 may beconfigured to select for the device 177 a first measuring rate 176 a,and based on the second indication 174 b, the controller 175 may beconfigured to select for the device 177 a second measuring rate 176 b.Optionally, while operating according to the measuring rate 176 a, thedevice 177 measures the affective response of the user 114 to the firstsegment 102 a. Optionally, while operating according to the secondmeasuring rate 176 b, the device 177 measures the affective response ofthe user 114 to the second segment 102 b.

In one embodiment, the semantic analyzer 173 utilizes a prediction ofemotional response to a segment of content, and provides in acorresponding indication one or more values related to the prediction.Optionally, to make the prediction, the semantic analyzer 173 utilizes apersonalized model that provides indications optimized for the user 114.Alternatively, to make the prediction, the semantic analyzer utilizes ageneral model that is not personalized for the user 114.

In one embodiment, the indication 174 a and/or the indication 174 b maybe interpreted as operating instructions for another module in a system,such as the controller 175 and/or the device 177. For example, theindications may be interpreted as an instruction to turn the device 177on or off, and/or to select a specific measuring rate for the device177. In another example, the indications may indicate how long thedevice 177 should be operated and/or specifies a period of time (e.g.,by providing start and end times) in which a device 177 should operate.

In some embodiments, the first indication 174 a indicates whether afirst value, related to a prediction of emotional response to the firstsegment 172 a, reaches a first predetermined threshold. Additionally,the second indication 174 b may indicate whether a second value, relatedto a prediction of emotional response to the second segment 172 b,reaches a second predetermined threshold. Optionally, the prediction ofemotional response to the first segment 172 a is generated as part ofthe analysis of the first segment 172 a by the semantic analyzer 173.Optionally, the prediction of emotional response to the second segment172 b is generated as part of the analysis of the second segment 172 bby the semantic analyzer. Optionally, a content emotional responsepredictor (content ERP) is utilized to make the prediction of emotionalresponse to the first segment 172 a and/or the prediction of emotionalresponse to the second segment 172 b. Optionally, a content emotionalresponse analyzer (content ERA) is utilized by the semantic analyzer 173in its analysis of segments of content.

In some embodiments, a device, such as the device 177, is used tomeasure affective response of the user 114. Optionally, the deviceincludes a sensor and/or is a sensor. Optionally, the sensor may be aphysiological sensor (e.g., a sensor that measures heart rate, galvanicskin response, and/or brainwave activity), and/or a sensor that measuresthe user's behavior (e.g., a camera, and/or a motion detector).Optionally, the device may include additional components to the sensor,such as a memory, a processor, a battery, a transmitter, and/or areceiver. Optionally, the device may be coupled to a user.

In some embodiments, the device 177 may output measurements of theaffective response of the user 114. Optionally, the measurements areforwarded to other modules of the system such as the measurementEmotional Response Predictor (measurement ERP) 188. Optionally, some ofthe measurements of the device 177 may include raw measurement values(i.e., the values undergo very little transformation prior to theiroutput). Alternatively or additionally, the measurements may includeprocessed values such as values that are the result of filtering,normalization, signal processing, feature extraction, encryption, and/orcompression.

In some embodiments, a measuring rate the controller 175 selects, suchas the first rate 176 a and/or the second rate 176 b, has presetoperation parameters (such as voltage, resolution, sampling rate).Alternatively or additionally, the measuring rate may include certainoperation parameters that are computed by the controller according tothe received indication and/or other data that is available at the timeof computation.

In one embodiment, a measuring rate is characterized by one or more ofthe following aspects: (i) the number of times, per unit of measurementtime (e.g., per minute), in which a device is configured to takemeasurements; (ii) the volume of data, per unit of measurement time,that the device is configured to generate while measuring; and/or (iii)the amount of power (e.g., wattage), the device is configured to drawper unit of measurement time.

It is to be noted that the phrase “per unit of measurement time” refersto a period of time in which the device may be operated and used tomeasure affective response. In this phrase, “measurement time” refers toa slot of time, during which the user may be measured with the device;however, in some cases, the system may elect not to measure the userwith the device (e.g., by keeping it off or in standby mode). In oneexample, a unit of measurement time may be one second or one minute.

Normalizing values according to a unit of measurement time enables thecomparison of amounts of measurements. For example, referring to thenumber of measurements per unit of measurement time enables thecomparison between the number of measurements taken by differentdevices, or measurements of the same device during different durations,and/or while operating at different measuring rates.

In one embodiment, a unit of measurement time may refer to a contiguousperiod of time in which the device may operate. Thus, for example, if ina fixed period of time, such as one second, the device (or a componentof the device) may rapidly alternate between being turned on and off, sothat half of the time it is on and half of the time it is off, themeasurement time of the device is considered to be the full second.

In some embodiments, a measuring rate for the device 177 is selectedfrom a set comprising at least the first and measuring rate 176 a andthe second measuring rate 176 b. While operating at the first measuringrate 176 a, the device 177 takes significantly fewer measurements ofaffective response, per unit of measurement time, compared to the numberof measurements taken by the device 177 while operating at the secondmeasuring rate 176 b. Additionally or alternatively, while operating atthe first measuring rate 176 a, the device 177 may generate less dataand/or consume less power, per unit of measurement time, compared towhen it operates at the second measuring rate.

Consequently, by selecting a measuring rate with respect to theinformation in indications, over time, the system 170 may reduce thetotal number of measurements taken by the device 177, the total amountof data generated by the device 177, and/or the total power consumed bythe device 177. Additionally, this may affect other modules in thesystem, such as reducing the power consumption of other elements thatparticipate in the processing of the affective response measurements,such as a processor that processes measurement data.

A device, such as the device 177, may operate according to differentmeasuring rates, which can dramatically change the number ofmeasurements the device is configured to take, the amount of data thedevice generates, and/or its power consumption. For example, thecontroller can cause the device to change from an operable mode to asubstantially inoperable mode (or standby mode), in which it takessignificantly fewer measurements, and vice versa. It is to be noted thata term such as “significantly less” or “significantly lower”, whenreferring to values such as a number of measurements taken, a volume ofdata generated, and/or power consumed, refers to a difference greaterthan a factor of two between the lower and higher values being compared.To illustrate this point, consider a device 177 capable of operating attwo measuring rates: A and B. When it is stated that the device takessignificantly fewer measurements while operating at measuring rate A,compared to the measurements the device takes while operating atmeasuring rate B, it means that the number of measurements taken by thedevice, per unit of measurement time, while operating in measuring rateA, is less than half of the number of measurements taken by the device,per unit of measurement time, while operating at measuring rate B.Alternatively or additionally, it may also mean that the volume ofmeasurement data generated by the device, per unit of measurement time,while operating in measuring rate A, is less than half of the volume ofmeasurement data generated by the device, per unit of measurement time,while operating at measuring rate B. And also, alternatively oradditionally, it may mean that the amount of power consumed by thedevice, per unit of measurement time, while operating at measuring rateA, is less than half of the amount of power consumed by the device, perunit of measurement time, while operating at measuring rate B.

In one example, an indication generated by the semantic analyzer 173includes a value that is indicative of a predicted interest level of auser in a segment of content. The controller 175 may select, accordingto the predicted interest level, parameters that define a measuringrate, and/or sample size such as a resolution of images captured by acamera. If the indication indicates that the predicted interest level islow, the controller may select the first measuring rate 176 a that maybe interpreted by a camera to capture low resolution images (e.g.,320×240 pixels). However, if the indication indicates the predictedinterest level is high, the controller may select a second measuringrate that may be interpreted by the camera to capture high-resolutionimages (e.g., 1920×1080 pixels). Additionally, if an indicationindicates that the interest is low, the first measuring rate 176 a mayconfigure the camera to record video images of the user at a low framerate. However, if the interest level is predicted to be high, the secondmeasuring rate 176 b can be selected, which may be interpreted by thecamera as instructions to records images at a much higher frame rate. Ahigher frame rate may enable more thorough analysis, such asidentification of briefly lasting micro-expressions, which may helpdetermine if the user was really interested in the content. A higherframe rate also corresponds to more measurements being taken per second,and a larger volume of image data being generated per second.

In one embodiment, as part of its analysis of the segment of content,the semantic analyzer 173 may predict a value describing the expectedemotional response of a user to the segment of content. An indication,generated by the semantic analyzer 173, may include informationregarding the expected emotional response, which is conveyed to thecontroller 175. Additionally or alternatively, the indication may beindicative of whether the expected emotional response reaches apredefined threshold; and the controller 175 may select a measuring ratebased on whether or not the predefined threshold is reached.

In one example, in the course of its analysis, the semantic analyzer 173predicts how excited a user is likely to get from reading a certain poston a social network site. An indication that is generated by thesemantic analyzer 173, indicates on a scale of 1-10 how excited a useris expected to be (10 being the extremely excited). The controller 175receives the indication, and with respect to the indication, selects ameasuring rate for an EEG head-mounted, battery operated, sensor thatmay be used to measure the affective response of the user 114 whilereading content. In order to save power, the system may elect to measurethe affective response of the user with the EEG sensor, while the userinteracts with the social network site, if the emotional response isexpected to reach a predefined threshold level, such as excitement of atleast 3 on the scale of 1-10. Measuring the user with the EEG may helpdetermine to what extent the user was really excited by what the userread. However, if the predicted value is too low, it is not likely thatthe clip is going to scare the user at all, so the system chooses not towaste resources on confirming that. Thus, if the expected emotionalresponse is below the threshold, the controller 175 may select a“reduced” measuring rate for the device 177, in which the device 177measures less. For example, the device may be configured to sample fewerbrainwave channels, take less frequent measurements, and/or configureits electrodes to draw less power each measurement. However, if theexpected emotional response reaches the threshold, the controller 175may select a “regular” measuring rate for the device 177, in which thedevice 177 performs significantly more measurements, compared to thereduced measuring rate. For example, the device 177 may be configured tomeasure more brainwave channels, take measurements more frequently,and/or set its electrodes to draw more power during each measurement.

In one embodiment, a measuring rate for the device 177 may be selectedby the controller 175 implicitly or explicitly. For example, an implicitselection of the measuring rate may occur when the controller 175 doesnot receive an indication from the semantic analyzer 173, when theindication has not been changed since the last signaling, and/or whenthe indication is the default indication. An explicit selection of themeasuring rate may occur when indication determines the measuring rateselected by the controller 175, e.g., by providing parameters describingthe measuring rate.

In one embodiment, the system 170 includes a measurement EmotionalResponse Predictor (measurement ERP) 188 configured to predict theuser's emotional response from data comprising measurements of affectiveresponse. Optionally, the measurements are processed prior to being sentto the measurement ERP 188 for prediction. For example, the measurementsmay undergo filtering and/or feature extraction. Additionally, themeasurement ERP 188 may be provided with information generated by thesemantic analyzer's 173 analysis of the segment of content 172, such asthe indications 174 a and/or 174 b. The measurement ERP 188 may utilizethis information for generating its prediction of the user's emotionalresponse.

In one embodiment, the system 170 further includes a measurement ERPtrainer 186. Optionally, the measurement ERP trainer 186 is providedwith information regarding an expected emotional response to segment ofcontents. For example, the measurement ERP trainer is provided with anindication 174 b generated by the semantic analyzer 173 as a result ofits analysis of the segment of content 172 b. The information providedto the measurement ERP trainer may be used by it to generate a label fora sample utilized by the measurement ERP trainer when training a modelfor a measurement ERP. Additionally or alternatively, the measurementERP trainer may be provided with measurements of affective response tothe user 114 to a segment of content, which may serve as a trainingsample for training a predictor. Optionally, the measurements may beprocessed prior to being sent to the measurement ERP trainer. Forexample, the measurements may undergo filtering and/or featureextraction in order to produce feature values that are included in oneor more training samples utilized by the module.

FIG. 19 illustrates one embodiment of a system configured to enable adevice 196 to measure affective response at varying measuring ratesbased on semantic analysis of content. The system includes at least areceiver 192, a processor 193, and a controller 195.

The receiver 192 is configured to receive a first indication 191 a and asecond indication 191 b derived from evaluations comprising semanticanalysis of first and second segments of content, respectively. Thesegments of content include data that represents text (e.g., thesegments include portions of text or media that may be converted totext). Optionally, the semantic analyzer 173 generates the indications191 a and 191 b by analyzing segments of content 172 a and 172 b,respectively. Optionally, the semantic analyzer 173 utilizes semanticanalysis methods in its analysis of the segments. The indications 191 aand 191 b may be received separately by the receiver 192 (e.g., each iscommunicated in a separate communication), or may be received jointly(e.g., both are communicated in the same communication). Optionally, thefirst indication 191 a and/or the second indications 191 b are functionsof the expected emotional responses to the first and second segments.For example, the indications may indicate the type of emotional responsea user is expected to have to content, the magnitude of the expectedemotional response to the content, and/or the expected benefit fromdetermining the emotional response to the content by measuring theaffective response of the user.

The first indication 191 a indicates that a first value, which is afunction of emotional response the first segment, does not reach a firstpredetermined threshold. Additionally, the second indication 191 bindicates that a second value, which is a function of emotional responseto the second segment, does reach a second predetermined threshold.

The processor 193 is configured to select, based on an indicationreceived by the receiver 192, a measuring rate for the device 196 to beused to measure affective response of a user 114 to segments of content.In one example, based on the first indication 191 a, the processor 193selects a first measuring rate 194 a for the device 196 to measureaffective response of the user to the first segment of content. Andbased on the second indication 191 b, the processor 193 selects a secondmeasuring rate 194 b for the device 196 to measure affective response ofthe user to the second segment.

The controller 195 is operable to configure the device 196 to measureaffective response of the user according to a measuring rate selected bythe processor 193, such as the first and second measuring rates. In oneexample, the controller 195 is configured to manage power consumption ofthe device 196 according to the measuring rate selected by the processor193. In another example, the controller 195 is configured to manage thenumber of measurements taken by the device 196, per unit of measurement(e.g., frame rate of video images). In yet another example, thecontroller is configured to manage the volume of data generated by thedevice 196, per unit of measurement time, for example by selectingnumber of brainwave channels to be monitored each second by a brainwavemonitor.

In one embodiment, the device 196 takes significantly fewermeasurements, per unit of measurement time, while operating at the firstmeasuring rate 194 to measure the affective response of the user 114 tothe first segment, compared to the number of measurements the device 196takes while operating at the second measuring rate 196 b to measure theaffective response of the user 114 to the second segment.

FIG. 20 illustrates one embodiment of a method for utilizing semanticanalysis to set a measuring rate of a device that measures affectiveresponse. The method includes the following steps:

In step 180, receiving a first indication derived from an evaluationinvolving semantic analysis of a first segment of content. The firstsegment includes data representing text.

In step 181, determining that the first indication indicates that afirst value related to a prediction of emotional response to the firstsegment does not reach a first predetermined threshold.

In step 182, configuring the device to measure, at a first measuringrate, affective response of a user to the first segment.

In step 183, receiving a second indication derived from an evaluationinvolving semantic analysis of a second segment of content. The firstsegment includes data representing text.

In step 184, determining that the second indication indicates that asecond value related to a prediction of emotional response to the secondsegment reaches a second predetermined threshold.

And in step 185, configuring the device to measure, at a secondmeasuring rate, affective response of the user to the second segment.

In one embodiment, the device takes significantly fewer measurements ofaffective response, per unit of measurement time, while operating at thefirst measuring rate, compared to number of measurements of affectiveresponse the device takes while operating at the second measuring rate.

In one embodiment, the first and second segments include datarepresenting text. The analysis of the first and second segmentsutilizes semantic analysis of the data representing text to produce theindications that indicate whether or not the predetermined threshold isreached. Additionally, the semantic analysis may be used to generatefeature values that describe the meaning of the text of the first andsecond segments. Optionally, the semantic analysis of the second segmentof content, which contributed to generating the second indication, isperformed less than one minute, or even one second, before the user isexposed to the second segment of content. For example, the secondsegment of content includes portions of messages sent to a user (e.g.,posts to a social network and/or instant message texts). Additionally oralternatively, the semantic analysis of the first segment of content,which contributed to generating the first indication, may be performedless than one minute, or even one second, before the user is exposed tothe first segment of content.

In another embodiment, semantic analysis of a segment of content isperformed substantially while the user 114 is exposed to the segmentand/or shortly after the user is exposed to it. For example, thesegments of content may include portions of a live conversation (e.g.,video, voice, and/or text), and the semantic analyzer performs itsanalysis of the segments substantially as conversation takes place.

In one embodiment, configuring the operation of the device according tothe first and second measuring rates may be done essentiallysimultaneously. For example, based on the first and second indications,the device may be programmed to measure at the first or second measuringrates at different times in the future, such as setting the sensor'smeasuring rate a priori to measure affective response to varioussegments of content before the segments of content are presented.Alternatively, the device may be configured to operate at differentmeasuring rate as indications are received. For example, each indicationmay indicate a certain measuring rate (and possibly operating times forthe device), which may override previous settings that were made by anindication that was received earlier.

FIG. 21 is a schematic illustration of an embodiment comprising a systemthat includes a computer game 179 that outputs segments of content 172and context information 178 about the segments 172. Optionally, the user114 provides the interactive computer game 179 with input thatinfluences events and/or characters in the computer game. Optionally,the segments of content 172 are displayed to the user 114 and/or areprovided to the semantic analyzer 173, possibly after undergoing someform of processing (e.g., feature extraction). Optionally, the contextinformation 178 is provided to: (i) the semantic analyzer 173 to assistwith the analysis of the segment 172, and/or (ii) the measurement ERP188 to improve its performance.

In one example, the computer game provides the semantic analyzer 173with context information regarding an event in the game that is relatedto the segment of content 172 a and/or 172 b. For instance, the computergame may provide the semantic analyzer with information indicative ofthe fact that the user 114 has been playing the same level for a longtime, which may indicate that the user might be getting bored and/orfrustrated with the game. In another example, the computer game providesthe semantic analyzer 173 with information regarding the status ofevents in the game not represented in the analyzed content (such as thestate of other characters in the game, and/or how the user 114 standswith respect to completing a mission). This information can providecontext and assist the semantic analyzer in determining the user'sexpected emotional response to the content, and thus improve theaccuracy of selection of appropriate measuring rates for measuring theaffective response to different segments of content. In yet anotherexample, the computer game includes a character whose actions arecontrolled at least partially by the user, and the information providedby the game to the semantic analyzer 173 is related to the actions ofthe character and/or to the fate of character in the game.

In one embodiment, the functionality of the semantic analyzer 173 isincluded in the interactive computer game 179. In this embodiment, thecomputer game provides an indication to the controller 175 that selectsa measuring rate for the device 177. In another embodiment, thefunctionality of both the semantic analyzer 173 and the controller 175are included in the interactive computer game 179. In this case, thehardware of the computer game may control the device 177.

In one embodiment, the controller 175 may be implemented as part of thesemantic analyzer 173. In another example, the controller 175, thedevice 177, and the semantic analyzer 173 are combined in a singledevice. Optionally, the device is battery powered (e.g., a smart phone).

In one embodiment, a segment of content may include visual content, suchas images and/or video. A content delivery module may be used to exposethe segments to the user 114. Optionally, the device 177 has at leastone component that is not shared with the content delivery module.Additionally or alternatively, the device 177 may include at least onecomponent that is shared with the content delivery module. Additionallyor alternatively, the device 177 may be a battery powered sensor, andthe content delivery module and the battery powered sensor are realizedin the same device.

In one embodiment, at least one element of the controller 175 isimplemented as a dedicated hardware element. For example, the controller175 may be a component coupled to the device 177. In another example,the controller 175 is coupled to a processor that processes affectiveresponse measurements.

In one embodiment, the controller 175 is implemented in software runningon a processor. In some cases, the controller may run, at least in part,on a device belonging to the user, such as a mobile phone. In somecases, the controller may run, at least in part, on a system remote ofthe user, such as on a cloud-based server.

In one embodiment, the controller 175 is realized as hardware that isembedded in the housing of the device 177. For example, the controllerfor a mobile EEG measuring headset may be embedded in the headsetitself, and communicate data wirelessly.

In another embodiment, the controller 175 is realized as software thatruns on a processor that belongs to the device 177. For example, thecontroller for a mobile EEG measuring headset may run on a processorthat belongs to the headset and is used to process EEG measurements.

In one embodiment, the controller 175 and/or the semantic analyzer 173are implemented at least in part, as software running on a remoteprocessor, such as a cloud-based server. For example, the cloud-basedservice can be used to analyze content that is to be presented to theuser 114, such as video to be streamed to the user's display. Accordingto results of the semantic analyzer's analysis, the controller 175running on the cloud-based service may send different operatinginstructions to the device 177 regarding how to measure the user.

In one embodiment, a system configured to use semantic analysis tomeasure affective response at varying measuring rates, comprising: asemantic analyzer configured to: receive a first segment of content,analyze the first segment utilizing semantic analysis, and output afirst indication that a first value related to a predicted emotionalresponse to the first segment does not reach a first predeterminedthreshold; wherein the first segment comprises data representing text;and a controller configured to select, based on the first indication, afirst measuring rate for a device for measuring affective response of auser to the first segment; the semantic analyzer is further configuredto: receive a second segment of content, analyze the second segmentutilizing semantic analysis, and output a second indication that asecond value related to a predicted emotional response to the secondsegment does reach a second predetermined threshold; wherein the secondsegment comprises data representing text; the controller is furtherconfigured to select, based on the second indication, a second measuringrate for the device for measuring affective response of the user to thesecond segment; wherein the device takes significantly fewermeasurements of affective response, per unit of measurement time, whileoperating at the first measuring rate, compared to number ofmeasurements of affective response the device takes while operating atthe second measuring rate. Optionally, the first and secondpredetermined thresholds represent first and second confidence levels inpredictions of emotional response, respectively; and wherein the firstand second values represent confidence in predictions of emotionalresponse to the first and second segments, respectively; whereby thefirst measuring rate is selected when the first indication indicatesthat confidence in a prediction of emotional response to the firstsegment does not reach the first confidence level, and the secondmeasuring rate is selected when the second indication indicates thatconfidence in a prediction of emotional response to the second segmentdoes reach the second confidence level. Optionally, the first and secondpredetermined thresholds represent first and second magnitudes of changein affective response, respectively; and wherein the first and secondvalues represent predicted changes in affective response due to exposureto the first and second segments, respectively; whereby the firstmeasuring rate is selected when the first indication indicates that apredicted change in affective response due to exposure to the firstsegment does not reach the first magnitude, and the second measuringrate is selected when the second indication indicates that a predictedchange in affective response due to exposure to the second segment doesreach the second magnitude. Optionally, the first and secondpredetermined thresholds represent first and second benefit levels,respectively; and wherein the first and second values representpredicted benefits to measuring affective response to the first andsecond segments, respectively; whereby the first measuring rate isselected when the first indication indicates that a predicted benefit ofmeasuring affective response to the first segment does not reach thefirst benefit level, and the second measuring rate is selected when thesecond indication indicates that a benefit of measuring affectiveresponse to the second segment does reach the second benefit level.Optionally, the first measuring rate is substantially zero, and whenoperating at the first measuring rate the device substantially does nottake affective response measurements of the user. Optionally, the deviceis one or more of the following: a physiological sensor, and an imagecapturing device. Optionally, the system further comprises a moduleconfigured to train an emotional response predictor (measurement ERP)based on feature values that are a product of measurements of theaffective response of the user to the second segment of content, and alabel based on the second indication generated by the semantic analyzer.Optionally, the semantic analyzer applies statistics on words that arecorrelated with certain emotional responses. Optionally, the semanticanalyzer applies Latent Semantic Analysis in order to associate asegment of content with a likely subject. Optionally, the semanticanalyzer evaluates the second segment shortly before the user is exposedto the second segment, whereby shortly is less than one minute.Optionally, the second segment is received from an interactive computergame that provides context information about the second segment that maybe utilized by the semantic analyzer to compute the second indication.

In one embodiment, a method for utilizing semantic analysis to set ameasuring rate of a device that measures affective response, comprising:receiving a first indication derived from an evaluation comprisingsemantic analysis of a first segment of content; wherein the firstsegment comprises data representing text; determining that the firstindication indicates that a first value related to a prediction ofemotional response to the first segment does not reach a firstpredetermined threshold, and configuring the device to measure, at afirst measuring rate, affective response of a user to the first segment;receiving a second indication derived from an evaluation comprisingsemantic analysis of a second segment of content; wherein the secondsegment comprises data representing text; and determining that thesecond indication indicates that a second value related to a predictionof emotional response to the second segment reaches a secondpredetermined threshold, and configuring the device to measure, at asecond measuring rate, affective response of the user to the secondsegment; wherein the device takes significantly fewer measurements ofaffective response, per unit of measurement time, while operating at thefirst measuring rate, compared to number of measurements of affectiveresponse the device takes while operating at the second measuring rate.Optionally, the semantic analysis of the data derived from the secondsegment is performed shortly before the user is exposed to the secondsegment, whereby shortly is less than one minute. Optionally, the firstand second indications indicate confidence in predictions of emotionalresponse to the first and second segments, respectively; whereby basedon the first indication, it is determined that confidence in theprediction of the emotional response to the first segment does not reachthe first predetermined threshold, and based on the second indication,it is determined that confidence in the prediction of the emotionalresponse to the second segment does reach the second predeterminedthreshold. Optionally, the first and second indications indicateexpected magnitudes of change in affective response due to exposure tothe first and second segments, respectively; whereby based on the firstindication, it is determined that expected magnitude of change inaffective response due to exposure to the first segment does not reachthe first predetermined threshold, and based on the second indication,it is determined that expected magnitude of change to affective responsedue to exposure to the second segment does reach the secondpredetermined threshold. Optionally, the first and second indicationsindicate how beneficial it is to measure the affective response to thefirst and second segments, respectively; whereby based on the firstindication, it is determined that benefit from measuring the affectiveresponse to the first segment does not reach the first predeterminedthreshold, and based on the second indication it is determined thatbenefit from measuring affective response to the second segment doesreach the second predetermined threshold. Optionally, the firstmeasuring rate is substantially zero, and substantially no measuring ofthe affective response of the user to being exposed to the first segmenttakes place. Optionally, the device comprises a battery poweredphysiological sensor, and measuring the affective response of the userto the second segment comprises utilizing the device to take aphysiological measurement of the user. Optionally, the device is part ofa battery powered image capturing device, and measuring the affectiveresponse of the user to the second segment comprises utilizing the imagecapturing device to capture an image of the user.

In one embodiment, a system configured to enable a device to measureaffective response at varying measuring rates based on semantic analysisof content, comprising: a receiver configured to receive first andsecond indications derived from evaluations comprising semantic analysisof first and second segments of content, respectively; wherein the firstand second segments comprise data representing text; and wherein thefirst indication indicates that a first value, which is a function ofemotional response the first segment, does not reach a firstpredetermined threshold, and the second indication indicates that asecond value, which is a function of emotional response to the secondsegment, does reach a second predetermined threshold; a processorconfigured to select, based on the first and second indications, firstand second measuring rates for measuring affective response,respectively; and a controller operable to configure the device tomeasure the affective response of a user according to the first andsecond measuring rates; wherein the device takes significantly fewermeasurements, per unit of measurement time, while operating at the firstmeasuring rate to measure affective response of the user to the firstsegment, compared to number of measurements the device takes whileoperating at the second measuring rate to measure affective response ofthe user to the second segment.

FIG. 22 illustrates one embodiment of a system 200 configured toconfigured to utilize semantic analysis to set a processing level forprocessing affective response measurements. The system includes at leasta semantic analyzer 173, a controller 204, and a processor 206.

In one embodiment, the semantic analyzer 173 is configured to: (i)receive a first segment 172 a of content and a second segment 172 b ofcontent, (ii) analyze the segments 172 a and 172 b utilizing semanticanalysis, and (iii) output a first indication 203 a and a secondindication 203 b. The first indication 203 a indicates that a firstvalue related to a prediction of emotional response to the first segment172 a does not reach a first predetermined threshold. The secondindication 203 b indicates that a second value related to a predictionof emotional response to the second segment 172 b does reach a secondpredetermined threshold.

In one embodiment, the semantic analyzer 173 may generate the firstindication 203 a and the second indication 203 b, which are conveyed tothe controller 204. The controller 204 is configured to select aprocessing level for the processor 206, based on a received indication.For example, based on the indication 203 a, the controller 204 isconfigured to select for the processor 206 a first processing level 205a, and based on the second indication 203 b, the controller 204 isconfigured to select for the processor 206 a second processing level 205b. Optionally, while operating at the first processing level 205 a, theprocessor 206 processes a first set of measurements 207 a of affectiveresponse of the user 114 taken with a device 208. Optionally, whileoperating at the second processing level 205 b, the processor 206processes a second set of measurements 207 b of affective response ofthe user 114 taken with the device 208.

In one embodiment, the indication 203 a and/or 203 b may be interpretedas operating instructions for another module in the system, such as thecontroller 204 and/or the processor 206. For example, the indicationsmay be interpreted as an instructions to the processor 206 to run acertain procedure and/or as an instruction to the controller 204 toselect a specific processing level for the processor 206. In anotherexample, the indications may indicate how long the processor 206 shouldprocess data and/or specifies a period of time (e.g., by providing startand end times) in which the processor 206 should process data.

In some embodiments, a device, such as the device 208, is used tomeasure affective response of the user 114. Optionally, the deviceincludes a sensor and/or is a sensor. Optionally, the sensor may be aphysiological sensor (e.g., a sensor that measures heart rate, galvanicskin response, and/or brainwave activity), and/or a sensor that measuresthe user's behavior (e.g., a camera, and/or a motion detector).Optionally, the device may include additional components to the sensor,such as a memory, a processor, a battery, a transmitter, and/or areceiver. Optionally, the device may be coupled to a user.

In some embodiments, the device 208 may output measurements of theaffective response of the user 114. For example, the device 208 mayoutput the measurements 207 a, taken essentially while the user 114 wasexposed to the first segment 172 a, which describe the affectiveresponse of the user 114 to being exposed to the first segment 172 a.Similarly, the device 208 may output the measurements 207 b, takenessentially while the user 114 was exposed to the second segment 172 a,which describe the affective response of the user 114 to being exposedto the second segment 172 b. Optionally, the measurements 207 a and/orthe measurements 207 b are forwarded to the processor 206 in order to beprocessed. Alternatively or additionally, the measurements 207 a and/orthe measurements 207 b may be forwarded to a module of the system suchas the measurement Emotional Response Predictor (measurement ERP).Optionally, some of the measurements the measurements 207 a and/or themeasurements 207 b may include raw measurement values (i.e., the valuesundergo very little transformation prior to their output). Alternativelyor additionally, the measurements 207 a and/or the measurements 207 bmay include processed values such as values that are the result offiltering, normalization, signal processing, feature extraction,encryption, and/or compression.

In some embodiments, the controller 204 is configured to utilize theindications 203 a and/or 203 b to select a processing level for theprocessor 206, from a set comprising a first processing level 205 a anda second processing level 205 b. The processing level may have presetoperation parameters (such as procedures to run, duration of processing,and/or volume of data to process). Alternatively or additionally, theprocessing level may include certain operation parameters that arecomputed by the controller 204 using the indications 203 a and/or 203 b,and/or other data that is available at the time of computation.

In some embodiments, the processor 206 is configured to processmeasurements of affective response of the user 114, while operating at acertain processing level. The certain processing level may be selectedfrom a set that includes at least the first processing level 205 a andthe second processing level 205 b. The computational load of theprocessor may be expressed in terms of the number of computationalcycles the processor utilizes to process a volume unit of data.

In one embodiment, based on the first indication 203 a, the controller204 selects the first processing level 205 a, which has a lowcomputational load, at which the processor 206 should operate whileprocessing measurements of affective response of the user 114 to thefirst segment 172 a. In addition, based on the second indication 203 b,the controller 204 selects the second processing level 205 b, which hasa higher computational load, at which the processor 206 should operatewhile processing measurements of affective response of the user 114 tothe second segment 172 b. Consequently, per volume unit of measurementdata, the number of computation cycles utilized for processingmeasurements of affective response of a user to the first segment, issignificantly lower than number of computation cycles utilized forprocessing measurements of affective response of the user to the secondsegment.

FIG. 23 illustrates one embodiment of a measurement ERP trainer 232 thatis configured to train an emotional response model 233 based on samples230 and labels 231 that are a product of the semantic analyzer 173.Optionally, the measurement ERP trainer 232 is provided with informationregarding an expected emotional response to the segment of content. Forexample, the measurement ERP trainer is provided with an indication 203b generated by the semantic analyzer 173 as a result of its analysis ofthe segment of content 172 b. The information provided to themeasurement ERP trainer 232, may be used by it in order to generate alabel 231 for a sample 230 utilized by the measurement ERP trainer whentraining the model 233. Additionally or alternatively, the measurementERP trainer 232 may be provided with measurements of affective responseof the user 114 to a segment of content taken by the device 208; thesemeasurements may be utilized to generate the samples 230. For example,the measurement ERP trainer 232 may receive the measurements 207 b takenby the device 208 which describe the affective response of the user 114to the second segment of content 172 b. Optionally, the measurementsreceived by the measurement ERP trainer 232 may be processed prior tobeing sent to the measurement ERP trainer. For example, the measurementsmay undergo filtering and/or feature extraction in order to producefeature values that are included in one or more training samplesutilized by the module. Optionally, the processing of the measurementsis done, at least in part, by the processor 206.

FIG. 24 illustrates one embodiment of a measurement Emotional ResponsePredictor (measurement ERP) 235 configured to predict emotional response236 based on a sample 234. Optionally, the measurement ERP 235 utilizesthe model 233 in order to generate the predicted emotional response 236.Optionally, the sample 234 includes feature values that are a product ofmeasurements of the device 208. Optionally, the measurements areprocessed prior to being sent to the measurement ERP 235. For example,the measurements 207 b may undergo filtering and/or feature extractionbefore being provided to the measurement ERP 235. Optionally, theprocessing of the measurements 207 b is done, at least in part, by theprocessor 206. Additionally, the measurement ERP 235 may be providedwith information generated by the semantic analyzer's analysis of thesegment of content 172 b, such as the indication 203 b. The measurementERP 235 may utilize this information for generating its predictedemotional response 236.

FIG. 25 illustrates one embodiment of a method for utilizing semanticanalysis to set a processing level of a processor processing affectiveresponse measurements. The method includes the following steps:

In step 210, receiving a first indication derived from an evaluationinvolving semantic analysis of a first segment of content. The firstsegment includes data representing text.

In step 211, determining that the first indication indicates that afirst value related to a prediction of emotional response to the firstsegment does not reach a first predetermined threshold.

In step 212, configuring a processor to operate at a first processinglevel to process measurements of affective response of the user to thefirst segment.

In step 213, receiving a second indication derived from an evaluationinvolving semantic analysis of a second segment of content. The secondsegment includes data representing text.

In step 214, determining that the second indication indicates that asecond value related to a prediction of emotional response to the secondsegment does reach a second predetermined.

And in step 215, configuring the processor to operate at a secondprocessing level to process measurements of affective response of theuser to the second segment.

In one embodiment, per volume unit of measurement data, number ofcomputation cycles utilized by the processor to process, at the firstprocessing level, measurements of the affective response of the user tothe first segment, is significantly lower than number of computationcycles utilized by the processor to process, at the second processinglevel, measurements of the affective response of the user to the secondsegment.

FIG. 26 illustrates one embodiment of a system 220 configured to utilizesemantic analysis to set processing level for affective responsemeasurements. The system 220 includes at least a receiver 222, aselector 224, and a processor 226.

The receiver 222 is configured to receive a first indication 221 a and asecond indication 221 b derived from evaluations comprising semanticanalysis of first and second segments of content, respectively. Thesegments of content include data that represents text (e.g., thesegments include portions of text or media that may be converted totext). Optionally, the semantic analyzer 173 generates the indications221 a and/or 221 b by analyzing segments of content 172 a and 172 b,respectively. Optionally, the semantic analyzer 173 utilizes semanticanalysis methods in its analysis of the segments. The indications 221 aand 221 b may be received separately by the receiver 222 (e.g., each iscommunicated in a separate communication), or may be received jointly(e.g., both are communicated in the same communication). Optionally, thefirst indication 221 a and/or the second indications 221 b are functionsof the expected emotional responses to the first and second segments.For example, the indications may indicate the type of emotional responsea user is expected to have to content, the magnitude of the expectedemotional response to the content, and/or the expected benefit fromdetermining the emotional response to the content by measuring theaffective response of the user.

The first indication 221 a indicates that a first value, which is afunction of emotional response the first segment, does not reach a firstpredetermined threshold. Additionally, the second indication 221 bindicates that a second value, which is a function of emotional responseto the second segment, does reach a second predetermined threshold.

The selector 224 is configured to select, based on an indicationreceived by the receiver 222, a processing level for the processor 226to be used to measure affective response of a user 114 to segments ofcontent. In one example, based on the first indication 221 a, theselector 224 selects a first processing level 225 a for the processor226 to process measurements of affective response of the user to thefirst segment of content. Optionally, measurements of affective responseof the user to the first segment of content are measurements 227 a,taken by the device 228. Optionally, the measurements 227 a were takenof the user 114, essentially while the user 114 was exposed to thesecond segment 172 b. And based on the second indication 221 b, theselector 224 selects a second processing level 225 b for the processor226 to process measurements of affective response of the user to thesecond segment of content. Optionally, measurements of affectiveresponse of the user to the second segment of content are measurements227 b, taken by the device 228. Optionally, the measurements 227 b weretaken of the user 114, essentially while the user 114 was exposed to thesecond segment 172 b.

The processor 226 is configured to receive first and second measurementsof affective response of a user to the first and second segments,respectively. Optionally, the first measurements are the measurements227 a and the second measurements are the measurements 227 b. Theprocessor 226 is also configured to process the first measurements atthe first processing level, and to process the second measurements atthe second processing level. Consequently, per volume unit ofmeasurement data, the number of computation cycles utilized by theprocessor 226 to process the first measurements is significantly lowerthan the number of computation cycles utilized by the processor 226 toprocess the second measurements. Thus, operating, at least during someof the times at the first processing level reduces the computationalload on the processor 226.

In one embodiment, the system 200 that includes the computer game 179,illustrated in FIG. 21 that outputs segments of content and contextinformation about the segments. Optionally, the user 114 provides theinteractive computer game with input that influences events and/orcharacters in the computer game. Optionally, the segments of content aredisplayed to the user 114 and/or are provided to the semantic analyzer173, possibly after undergoing some form of processing (e.g., featureextraction). Optionally, the context information is provided to: (i) thesemantic analyzer 173 to assist with the analysis of a segment ofcontent, (ii) the measurement ERP trainer 232 to contribute in thegeneration of feature values related to the context, and/or (iii) themeasurement ERP 235 in order to be utilized for generating featurevalues related to the context that may improve the predictiveperformance.

In one embodiment, the functionality of the semantic analyzer 173 isincluded in the interactive computer game. In this embodiment, thecomputer game provides an indication to the controller 204 that selectsa processing level for the processor 206. In another embodiment, thefunctionality of both the semantic analyzer 173 and the controller 204are included in the interactive computer game. In this case, thehardware of the computer game may control the processor 206 and/or thedevice 208.

There may be various hardware configurations that involve the semanticanalyzer 173, the controller 204, and/or the processor 206. In oneexample, the controller 204 may be implemented, at least in part, ashardware that is part of the processor 206. In another example, thecontroller 204 may be implemented, at least in part, as software thatruns on the processor 206. In another example, the controller 204 may beimplemented as part of the semantic analyzer 173. In still anotherexample, the controller 204, the processor 206, and the semanticanalyzer 173 are combined in a single device. Optionally, the device isbattery powered (e.g., a smart phone). Optionally, the device 208 isalso part of the device and/or draws power from the device's powersupply.

In one embodiment, the processor 206 is embedded in a battery powereddevice, and the measurements of affective response of the user to thesecond segment include physiological measurements of the user taken withthe a physiological sensor. Additionally or alternatively, themeasurements of affective response of the user to the second segmentcomprise comprises images of the user taken with an image capturingdevice.

In one embodiment, the controller 204 and/or the semantic analyzer 173are implemented at least in part, as software running on a remoteprocessor, such as a cloud-based server. For example, a cloud-basedservice can be used to analyze content that is to be presented to theuser 114, such as video to be streamed to the user's display. Accordingto indications of the semantic analyzer 173, the controller 204, runningon the cloud-based service, may send different operating instructions tothe processor 206, such as specifying a processing level for theprocessor 206 at to operate while processing affective responsemeasurements of the user.

In one embodiment, a segment of content may include visual content, suchas images and/or video. A content delivery module may be used to exposethe segments to the user 114. Optionally, the device 208 has at leastone component that is not shared with the content delivery module.Additionally or alternatively, the device 208 may include at least onecomponent that is shared with the content delivery module. Additionallyor alternatively, the device 208 may be a battery powered sensor, andthe content delivery module and the battery powered sensor are realizedin the same device.

In one embodiment, a system configured to utilize semantic analysis toset a processing level for processing affective response measurements,comprising: a semantic analyzer configured to: receive a first segmentof content, analyze the first segment utilizing semantic analysis, andoutput a first indication that a first value related to a predictedemotional response to the first segment does not reach a firstpredetermined threshold; wherein the first segment comprises datarepresenting text; and a controller configured to select, based on thefirst indication, a first processing level for a processor to processmeasurements of affective response of a user to the first segment; thesemantic analyzer is further configured to: receive a second segment ofcontent, analyze the second segment utilizing semantic analysis, andoutput a second indication that a second value related to a predictedemotional response to the second segment reaches a second predeterminedthreshold; wherein the second segment comprises data representing text;the controller is further configured to select, based on the secondindication, a second processing level for a processor to processmeasurements of affective response of the user to the second segment;wherein, per volume unit of measurement data, number of computationcycles utilized by the processor to process, at the first processinglevel, measurements of the affective response of the user to the firstsegment, is significantly lower than number of computation cyclesutilized by the processor to process, at the second processing level,measurements of the affective response of the user to the secondsegment. Optionally, the system further comprises a device operable toprovide measurements of the affective response of the user; wherein thedevice is one or more of the following: a physiological sensor, and animage capturing device. Optionally, the system further comprises amodule configured to train an emotional response predictor (measurementERP) based on feature values that are a product of measurements of theaffective response of the user taken by the device and labels that are aproduct of the semantic analyzer. Optionally, the system furthercomprises an emotional response predictor (measurement ERP) configuredto predict emotional response based on feature values that are a productof the affective response of the user taken by the device. Optionally,the first and second predetermined thresholds represent first and secondconfidence levels in predictions of emotional response, respectively;and wherein the first and second values represent confidence inpredictions of emotional response to the first and second segments,respectively; whereby the processor is configured to operate at thefirst processing level to process measurements of affective response ofthe user to the first segment when the first indication indicates thatconfidence in a prediction of emotional response to the first segmentdoes not reach the first confidence level, and the processor isconfigured to operate at the second processing level to processmeasurements of affective response of the user to the second segmentwhen the second indication indicates that confidence in a prediction ofemotional response to the second segment does reach the secondconfidence level. Optionally, the first and second predeterminedthresholds represent first and second magnitudes of change in affectiveresponse, respectively; and wherein the first and second valuesrepresent predicted changes in affective response due to exposure to thefirst and second segments, respectively; whereby the processor isconfigured to operate at the first processing level to processmeasurements of affective response of the user to the first segment whenthe first indication indicates that a predicted change in affectiveresponse due to exposure to the first segment does not reach the firstmagnitude, and the processor is configured to operate at the secondprocessing level to process measurements of affective response of theuser to the second segment when the second indication indicates that apredicted change in affective response due to exposure to the secondsegment does reach the second magnitude. Optionally, the first andsecond predetermined thresholds represent first and second benefitlevels, respectively; and wherein the first and second values representpredicted benefits to measuring affective response to the first andsecond segments, respectively; whereby the processor is configured tooperate at the first processing level to process measurements ofaffective response of the user to the first segment when the firstindication indicates that a benefit of measuring affective response tothe first segment does not reach the first benefit level, and theprocessor is configured to operate at the second processing level toprocess measurements of affective response of the user to the secondsegment when the second indication indicates that a benefit of measuringaffective response to the second segment does reach the second benefitlevel. Optionally, the semantic analyzer applies statistics on wordsthat are correlated with certain emotional responses. Optionally, thesemantic analyzer applies Latent Semantic Analysis in order to associatea segment of content with a likely subject. Optionally, the semanticanalyzer evaluates the second segment shortly before the user is exposedto the second segment, whereby shortly is less than one minute.Optionally, the second segment is received from an interactive computergame that provides context information about the second segment that maybe utilized by the semantic analyzer to compute the second indication.

In one embodiment, a method for utilizing semantic analysis to set aprocessing level of a processor processing affective responsemeasurements, comprising: receiving a first indication derived from anevaluation comprising semantic analysis of a first segment of content;wherein the first segment comprises data representing text; determiningthat the first indication indicates that a first value related to aprediction of emotional response to the first segment does not reach afirst predetermined threshold, and configuring a processor to operate ata first processing level to process measurements of affective responseof a user to the first segment; receiving a second indication derivedfrom an evaluation comprising semantic analysis of a second segment ofcontent; wherein the second segment comprises data representing text;and determining that the second indication indicates that a second valuerelated to a prediction of emotional response to the second segmentreaches a second predetermined threshold, and configuring the processorto operate at a second processing level to process measurements ofaffective response of the user to the second segment; wherein, pervolume unit of measurement data, number of computation cycles utilizedby the processor to process, at the first processing level, measurementsof the affective response of the user to the first segment, issignificantly lower than number of computation cycles utilized by theprocessor to process, at the second processing level, measurements ofthe affective response of the user to the second segment. Optionally,the first and second indications indicate confidence in predictions ofemotional response to the first and second segments, respectively;whereby based on the first indication, it is determined that confidencein the prediction of the emotional response to the first segment doesnot reach the first predetermined threshold, and based on the secondindication, it is determined that confidence in the prediction of theemotional response to the second segment does reach the secondpredetermined threshold. Optionally, the first and second indicationsindicate expected magnitudes of change in affective response due toexposure to the first and second segments, respectively; whereby basedon the first indication, it is determined that expected magnitude ofchange in affective response due to exposure to the first segment doesnot reach the first predetermined threshold, and based on the secondindication, it is determined that expected magnitude of change toaffective response due to exposure to the second segment does reach thesecond predetermined threshold. Optionally, the first and secondindications indicate how beneficial it is to measure the affectiveresponse to the first and second segments, respectively; whereby basedon the first indication, it is determined that benefit from measuringthe affective response to the first segment does not reach the firstpredetermine threshold, and based on the second indication it isdetermined that benefit from measuring affective response to the secondsegment does reach the second predetermined threshold. Optionally, theprocessor is embedded in a battery powered device, and the measurementsof affective response of the user to the second segment comprisephysiological measurements of the user taken with a physiologicalsensor. Optionally, the processor is embedded in a battery powereddevice, and the measurements of affective response of the user to thesecond segment comprise comprises images of the user taken with an imagecapturing device. Optionally, the method further comprises forwarding,to a module that trains a measurement Emotional Response Predictor(measurement ERP), a label corresponding to an expected emotionalresponse to being exposed to the second segment, and a sample comprisinga product of the measurements of the affective response of the user tothe second segment. Optionally, the method further comprises forwardinga product of the measurements of the affective response of the user tobeing exposed to the second segment to a measurement emotional responsepredictor.

In one embodiment, a system configured to utilize semantic analysis toset processing level for affective response measurements, comprising: areceiver configured to receive first and second indications derived fromevaluations comprising semantic analysis of first and second segments ofcontent, respectively; wherein the first and second segments comprisedata representing text; and wherein the first indication indicates thata first value, which is a function of emotional response the firstsegment, does not reach a first predetermined threshold, and the secondindication indicates that a second value, which is a function ofemotional response to the second segment, does reach a secondpredetermined threshold; a selector configured to select, based on thefirst and second indications, first and second processing levels,respectively, for processing measurements of affective response; and aprocessor configured to receive first and second measurements ofaffective response of a user to the first and second segments,respectively; the processor is further configured to process the firstmeasurements at the first processing level, and to process the secondmeasurements at the second processing level; wherein, per volume unit ofmeasurement data, the number of computation cycles utilized by theprocessor to process the first measurements is significantly lower thanthe number of computation cycles utilized by the processor to processthe second measurements.

Predetermined Thresholds

In some embodiments described in this disclosure, segments of contentare analyzed in order to predict an emotional response to the segments.For example, the analysis may be performed by a content ERA and/or asemantic analyzer. Optionally, analysis of a segment of content isperformed in order to determine whether a value related to an emotionalresponse emotional response to the segment of content reaches apredetermined threshold. Optionally, after analyzing the segment ofcontent, an indication is generated indicating whether value related tothe emotional response to the segment of content reaches thepredetermined threshold. Optionally, the prediction of emotionalresponse is a prediction of emotional response of the user 114. Forexample, the predicted emotional response of the user 114 is obtainedfrom an emotional response predictor personalized for the user 114.Alternatively, the predicted emotional response to the segment may bethe emotional response of another user (e.g., using a model of anotheruser), or a generic “representative user”. Optionally, based on whetherthe indication indicates that the predetermined threshold was reached,in different embodiments described herein, system components may beconfigured to operate in different ways.

In some embodiments, based on an indication, the controller 108, selectsa certain mode of operation for the device 112. Optionally, selectingthe certain mode of operation for the device 112 also involvesconfiguring the device 112 to operate in the certain mode of operation.In other embodiments, based on an indication, the controller 154 selectsfor the processor 156 a certain processing level at which to processmeasurements of affective response. Optionally, selecting the certainprocessing level also involves configuring the processor 156 to operateat the certain processing level. In still other embodiments, based on anindication, the controller 175 selects a certain measuring rate for thedevice 177. Optionally, selecting the certain measuring rate alsoinvolves configuring the device 177 to measure at the certain measuringrate. And in yet other embodiments, based on an indication, thecontroller 204 selects for the processor 206 a certain processing levelat which to process measurements of affective response. Optionally,selecting the certain processing level also involves configuring theprocessor 206 to process measurements of affective response at thecertain processing level.

In some embodiments, first and second segments of content are analyzed,and based on the analysis, first and second indications are generated.Optionally, the first indication indicates whether a value related to aprediction of emotional response to the first segment reaches a firstpredetermined threshold. Optionally, the second indication indicateswhether a value related to a prediction of emotional response to thesecond segment reaches a second predetermined threshold. It is to benoted that when stated that a value reaches a threshold, it is meantthat the value equals the threshold or exceeds it. Below are listedexamples of embodiments which describe different types of predeterminedthresholds and how they may be utilized.

In one embodiment, the first and second predetermined thresholdsrepresent first and second confidence levels for predictions ofemotional response, respectively. That is, the first predeterminedthreshold represents the first confidence level, and the secondpredetermined threshold represents the second confidence level.Optionally, the first predetermined threshold and the secondpredetermined threshold may be the same, in which case the bothrepresent the same confidence level. In this example, the first andsecond indications, indicate values representing confidence inpredictions of emotional response to the first and second segments,respectively. That is, the first indication indicates confidence in theprediction of emotional response to the first segment, and similarly thesecond indication indicates confidence in the prediction of emotionalresponse to the second segment. Optionally, a user's affective responsemay be processed and/or measured more extensively when the predictionsof emotional response to the segments of content are made with highconfidence. If the prediction of emotional response to a segment ofcontent is made with low confidence, the system may elect not to measurethe user's affective response to the segment, or to devote fewerresources for that purpose.

In one embodiment, based on the first indication 106 a, it is determinedthat confidence in the prediction of the emotional response to the firstsegment 102 a does not reach the first predetermined threshold.Accordingly, the first mode of operation 110 a is selected for thedevice 112 for measuring affective response of the user 114 to the firstsegment 102 a. Additionally, based on the second indication 106 b, it isdetermined that confidence in the prediction of the emotional responseto the second segment 102 b does reach the second predeterminedthreshold. Accordingly, the second mode of operation 110 b is selectedfor the device 112 for measuring affective response of the user 114 tothe second segment 102 b.

In another embodiment, based on the first indication 106 a, it isdetermined that confidence in the prediction of the emotional responseto the first segment 102 a does not reach the first predeterminedthreshold. Accordingly, the first processing level 155 a is selected forthe processor 156 to process measurements of affective response of theuser 114 to the first segment 102 a. Additionally, based on the secondindication 106 b, it is determined that confidence in the prediction ofthe emotional response to the second segment 102 b does reach the secondpredetermined threshold. Accordingly, the second processing level 155 bis selected for the processor 156 to process measurements of affectiveresponse of the user 114 to the second segment 102 b.

In yet another embodiment, based on the first indication 174 a, it isdetermined that confidence in the prediction of the emotional responseto the first segment 172 a does not reach the first predeterminedthreshold. Accordingly, the first measuring rate 176 a is selected forthe device 177 for measuring affective response of the user 114 to thefirst segment 172 a. Additionally, based on the second indication 174 b,it is determined that confidence in the prediction of the emotionalresponse to the second segment 172 b does reach the second predeterminedthreshold. Accordingly, the second measuring rate 176 b is selected forthe device 177 for measuring affective response of the user 114 to thesecond segment 172 b.

In still embodiment, based on the first indication 203 a, it isdetermined that confidence in the prediction of the emotional responseto the first segment 172 a does not reach the first predeterminedthreshold. Accordingly, the first processing level 205 a is selected forthe processor 206 to process measurements of affective response of theuser 114 to the first segment 172 a. Additionally, based on the secondindication 203 b, it is determined that confidence in the prediction ofthe emotional response to the second segment 172 b does reach the secondpredetermined threshold. Accordingly, the second processing level 205 bis selected for the processor 206 to process measurements of affectiveresponse of the user 114 to the second segment 172 b.

In another example, the first and second predetermined thresholdsrepresent first and second magnitudes of affective response and/orchange in affective response, respectively. That is, the firstpredetermined threshold represents the first magnitude, and the secondpredetermined threshold represents the second magnitude. Optionally, thefirst predetermined threshold and the second predetermined threshold maybe the same, in which case the both represent the same magnitude ofchange in affective response. In this example, the first and secondindications, indicate values representing expected magnitudes and/orexpected magnitude of change in affective response due to exposure tothe first segment and the second segments, respectively. The expectedmagnitude and/or magnitudes of change may be derived from predictions ofemotional response to the first and second segments. Optionally, thepredictions are performed by the content ERA, a semantic analyzer and/ora content ERP.

In one example, more power may be typically devoted to measure theuser's affective response to a segment of content when a there isexpected to be a noticeable change to the user's affective response;while in other cases, when the change to the user's affective responseis not predicted to be significant, the user's affective response may becursorily measured (e.g., takes noisy and/or sporadic measurements).

In another example, a user's affective response may be measured moreextensively when the user is expected to feel a certain type of emotion.For instance, a computer game may provide the user with segments ofcontent that may be intense, possibly frightening, action sequences.When a user is exposed to a segment, the user's affective response maybe measured in order to determine more precisely how much the segmentinfluenced the user, and possibly identify what elements in the contentinfluenced the user substantially. In this example, affective responseof the user to a segment may be measured (or measured more extensively)if analysis of the segment indicates that the user is expected to befrightened or excited by the segment (e.g., analysis of the segment mayreveal it to be intense and action packed). In this example, the firstand second indications may indicate whether the segments are predictedto frighten and/or excite the user. If it is determined that a segmentis likely not to frighten and/or excite the user, measurements may betaken more extensively and/or processed more extensively. In thisexample, the indications may indicate how discernable measurements ofaffective response, which reflect a certain emotion (e.g., fright), arelikely to be.

In one embodiment, based on the first indication 106 a, it is determinedthat expected magnitude of change in affective response due to exposureto the first segment 102 a does not reach the first predeterminedthreshold. Accordingly, the first mode of operation 110 a is selectedfor the device 112 for measuring affective response of the user 114 tothe first segment 102 a. Additionally, based on the second indication106 b, it is determined that expected magnitude of change to affectiveresponse due to exposure to the second segment 102 b does reach thesecond predetermined threshold. Accordingly, the second mode ofoperation 110 b is selected for the device 112 for measuring affectiveresponse of the user 114 to the second segment 102 b.

In another embodiment, based on the first indication 106 a, it isdetermined that expected magnitude of change in affective response dueto exposure to the first segment 102 a does not reach the firstpredetermined threshold. Accordingly, the first processing level 155 ais selected for the processor 156 to process measurements of affectiveresponse of the user 114 to the first segment 102 a. Additionally, basedon the second indication 106 b, it is determined that expected magnitudeof change to affective response due to exposure to the second segment102 b does reach the second predetermined threshold. Accordingly, thesecond processing level 155 b is selected for the processor 156 toprocess measurements of affective response of the user 114 to the secondsegment 102 b.

In yet another embodiment, based on the first indication 174 a, it isdetermined that expected magnitude of change in affective response dueto exposure to the first segment 172 a does not reach the firstpredetermined threshold. Accordingly, the first measuring rate 176 a isselected for the device 177 for measuring affective response of the user114 to the first segment 172 a. Additionally, based on the secondindication 174 b, it is determined that expected magnitude of change toaffective response due to exposure to the second segment 172 b doesreach the second predetermined threshold. Accordingly, the secondmeasuring rate 176 b is selected for the device 177 for measuringaffective response of the user 114 to the second segment 172 b.

In still another embodiment, based on the first indication 203 a, it isdetermined that expected magnitude of change in affective response dueto exposure to the first segment 172 a does not reach the firstpredetermined threshold. Accordingly, the first processing level 205 ais selected for the processor 206 to process measurements of affectiveresponse of the user 114 to the first segment 172 a. Additionally, basedon the second indication 1203 b, it is determined that expectedmagnitude of change to affective response due to exposure to the secondsegment 172 b does reach the second predetermined threshold.Accordingly, the second processing level 205 b is selected for theprocessor 206 to process measurements of affective response of the user114 to the second segment 172 b.

In another example, the first and second predetermined thresholdsrepresent first and second benefit levels, respectively. That is, thefirst predetermined threshold represents the first benefit level, andthe second predetermined threshold represents the second benefit level.Optionally, the first predetermined threshold and the secondpredetermined threshold may be the same, in which case the bothrepresent the same benefit level. In this example, the first and secondindications, indicate how beneficial it is to measure the affectiveresponse to the first and second segments, respectively. That is, thefirst indication indicates a benefit expected from measuring affectiveresponse of the user to the first segment, and the second indicationindicates a benefit expected from measuring affective response of theuser to the second segment.

In one embodiment, affective response of a user to segments of contentare measured more extensively and/or processed more extensively when itis beneficial to determine the user's emotional response. In oneexample, there may be a benefit to determine a user's emotional responseto content if there are opportunities to customize the content and/oroffer services to the user based on the user's emotional response. Inanother example, determining an emotional response of a user to contentmay be more beneficial if the corresponding measurement data is requiredto generate a model of the user (e.g., to train a content ERPpersonalized for the user). If the user already has a well-trainedmodel, it may be less beneficial to measure the user. However, if theuser has an incomplete model, or lacks a model altogether, it may bemore beneficial to measure the user in order to obtain training data totrain a model for the user.

In one embodiment, based on the first indication 106 a, it is determinedthat a benefit of measuring affective response to the first segment doesnot reach the first predetermined threshold. Accordingly, the first modeof operation 110 a is selected for the device 112 for measuringaffective response of the user 114 to the first segment 102 a.Additionally, based on the second indication 106 b, it is determinedthat a benefit from measuring affective response to the second segment102 b does reach the second predetermined threshold. Accordingly, thesecond mode of operation 110 b is selected for the device 112 formeasuring affective response of the user 114 to the second segment 102b.

In another embodiment, based on the first indication 106 a, it isdetermined that a benefit of measuring affective response to the firstsegment does not reach the first predetermined threshold. Accordingly,the first processing level 155 a is selected for the processor 156 toprocess measurements of affective response of the user 114 to the firstsegment 102 a. Additionally, based on the second indication 106 b, it isdetermined that a benefit from measuring affective response to thesecond segment 102 b does reach the second predetermined threshold.Accordingly, the second processing level 155 b is selected for theprocessor 156 to process measurements of affective response of the user114 to the second segment 102 b.

In yet another embodiment, based on the first indication 174 a, it isdetermined that a benefit of measuring affective response to the firstsegment 172 a does not reach the first predetermined threshold.Accordingly, the first measuring rate 176 a is selected for the device177 for measuring affective response of the user 114 to the firstsegment 172 a. Additionally, based on the second indication 174 b, it isdetermined that a benefit from measuring affective response to thesecond segment 172 b does reach the second predetermined threshold.Accordingly, the second measuring rate 176 b is selected for the device177 for measuring affective response of the user 114 to the secondsegment 172 b.

In still another embodiment, based on the first indication 203 a, it isdetermined that a benefit of measuring affective response to the firstsegment 172 a does not reach the first predetermined threshold.Accordingly, the first processing level 205 a is selected for theprocessor 2066 to process measurements of affective response of the user114 to the first segment 172 a. Additionally, based on the secondindication 203 b, it is determined that a benefit from measuringaffective response to the second segment 172 b does reach the secondpredetermined threshold. Accordingly, the second processing level 205 bis selected for the processor 206 to process measurements of affectiveresponse of the user 114 to the second segment 172 b.

Affective Response Measurements

In some embodiments, a device used to measure affective response of auser may include one or more sensors, be a sensor, and/or be part of asensor. Optionally, a sensor may include, without limitation, one ormore of the following: a physiological sensor, an image capturingdevice, a microphone, a movement sensor, a pressure sensor, and/or amagnetic sensor.

Herein, a “sensor” may refer to a whole structure housing a device usedfor measuring a physical property, or to one or more of the elementscomprised in the whole structure. For example, when the sensor is acamera, the word sensor may refer to the entire structure of the camera,or just to its CMOS detector.

A physiological signal is a value that reflects a person's physiologicalstate. Some examples of physiological signals that may be measuredinclude: Heart Rate (HR), Blood-Volume Pulse (BVP), Galvanic SkinResponse (GSR), Skin Temperature (ST), respiration, electrical activityof various body regions or organs such as brainwaves measured withelectroencephalography (EEG), electrical activity of the heart measuredby an electrocardiogram (ECG), electrical activity of muscles measuredwith electromyography (EMG), and electrodermal activity (EDA) thatrefers to electrical changes measured at the surface of the skin.

A person's affective response may be expressed by behavioral cues, suchas facial expressions, gestures, and/or other movements of the body.Behavioral measurements of a user may be obtained utilizing varioustypes of sensors, such as an image capturing device (e.g., a camera), amovement sensor, an acoustic sensor, an accelerometer, a magneticsensor, and/or a pressure sensor.

In one embodiment, images of the user are captured with an imagecapturing device such as a camera. In another embodiment, images of theuser are captured with an active image capturing device that transmitselectromagnetic radiation (such as radio waves, millimeter waves, ornear visible waves) and receives reflections of the transmittedradiation from the user. Optionally, captured images are in twodimensions and/or three dimensions. Optionally, captured images arecomprised of one or more of the following types: single images,sequences of images, video clips.

Affective response measurement data, such as the data generated by adevice used to measure affective response, may be processed in manyways. The processing of the affective response measurement data may takeplace before, during and/or after the data is stored and/or transmitted.Optionally, at least some of the processing of the data is performed bya sensor that participates in the collection of the measurement data.Optionally, at least some of the processing of the data is performed bya processor that receives the data in raw (unprocessed) form, orpartially processed form. There are various ways in which affectiveresponse measurement data may be processed in the different embodiments,some of them are described in the following embodiments and examples:

In some embodiments, at least some of the affective responsemeasurements may undergo signal processing, such as analog signalprocessing, discrete time signal processing, and/or digital signalprocessing.

In some embodiments, at least some of the affective responsemeasurements may be scaled and/or normalized. For example, themeasurement values may be scaled to be in the range [−1,+1]. In anotherexample, the values of some of the measurements are normalized toz-values, which bring the mean of the values recorded for the modalityto 0, with a variance of 1. In yet another example, statistics areextracted from the measurement values, such as statistics of theminimum, maximum, and/or various moments of the distribution, such asthe mean, variance, or skewness. Optionally, the statistics are computedfor measurement data that includes time-series data, utilizing fixed orsliding windows.

In some embodiments, at least some of the affective responsemeasurements may be subjected to feature extraction and/or reductiontechniques. For example, affective response measurements may undergodimensionality-reducing transformations such as Fisher projections,Principal Component Analysis (PCA), and/or feature subset selectiontechniques like Sequential Forward Selection (SFS) or SequentialBackward Selection (SBS).

In some embodiments, affective response measurements comprising imagesand/or video may be processed in various ways. In one example,algorithms for identifying cues like movement, smiling, laughter,concentration, body posture, and/or gaze, are used in order to detecthigh-level image features. Additionally, the images and/or video clipsmay be analyzed using algorithms and/or filters for detecting and/orlocalizing facial features such as location of eyes, brows, and/or theshape of mouth. Additionally, the images and/or video clips may beanalyzed using algorithms for detecting facial expressions and/ormicro-expressions.

In another example, images are processed with algorithms for detectingand/or describing local features such as Scale-Invariant FeatureTransform (SIFT), Speeded Up Robust Features (SURF), scale-spacerepresentation, and/or other types of low-level image features.

In some embodiments, processing affective response measurements involvescompressing and/or encrypting portions of the data. This may be done fora variety of reasons, for instance, in order to reduce the volume ofmeasurement data that needs to be transmitted. Another reason to usecompression and/or encryption is that it helps protect the privacy of ameasured user by making it difficult for unauthorized parties to examinethe data. Additionally, the compressed data may be pre-processed priorto its compression.

In addition, the literature describes various algorithmic approachesthat can be used for processing affective response measurements,acquired utilizing various types of sensors. Some embodiments mayutilize these known, and possibly other yet to be discovered, methodsfor processing affective response measurements. Some examples include:(i) a variety of physiological measurements may be preprocessedaccording to the methods and references listed in van Broek, E. L.,Janssen, J. H., Zwaag, M. D., D. M. Westerink, J. H., & Healey, J. A.(2009), Prerequisites for Affective Signal Processing (ASP), InProceedings of the International Joint Conference on BiomedicalEngineering Systems and Technologies, INSTICC Press, incorporated hereinby reference; (ii) a variety of acoustic and physiological signals maybe pre-processed and have features extracted from them according to themethods described in the references cited in Tables 2 and 4, Gunes, H.,& Pantic, M. (2010), Automatic, Dimensional and Continuous EmotionRecognition, International Journal of Synthetic Emotions, 1 (1), 68-99,incorporated herein by reference; (iii) Pre-processing of Audio andvisual signals may be performed according to the methods described inthe references cited in Tables 2-4 in Zeng, Z., Pantic, M., Roisman, G.,& Huang, T. (2009), A survey of affect recognition methods: audio,visual, and spontaneous expressions, IEEE Transactions on PatternAnalysis and Machine Intelligence, 31 (1), 39-58, incorporated herein byreference; and (iv) pre-processing and feature extraction of variousdata sources such as images, physiological measurements, voicerecordings, and text based-features, may be performed according to themethods described in the references cited in Tables 1, 2, 3, 5 in Calvo,R. A., & D'Mello, S. (2010). Affect Detection: An InterdisciplinaryReview of Models, Methods, and Their Applications. IEEE Transactions onaffective computing 1(1), 18-37, incorporated herein by reference.

In some embodiments, the duration in which devices operate in order tomeasure the user's affective response may differ depending on one ormore of the following: (i) the type of content the user is exposed to,(ii) the type of physiological and/or behavioral signal being measured,and (iii) the type of sensor utilized for the measurement. In somecases, the user's affective response to a segment of content may bemeasured by a device substantially continually throughout the period inwhich the user is exposed to the segment of content. However, in othercases, the duration during which the user's affective response to thesegment of content is measured need not necessarily overlap, or beentirely contained in the time in which the user is exposed to thesegment of content.

With some physiological signals, there is an inherent delay between thetime in which a stimulus occurs and changes the user's emotional state,and the time in which the corresponding affective response is observedvia a change in the physiological signal's measurement values. Forexample, an affective response comprising changes in skin temperaturemay take several seconds to be detected by a sensor. In addition, somephysiological signals may depart very rapidly from baseline values, buttake much longer to return to the baseline values.

In some cases, the physiological signal might change quickly as a resultof a stimulus, but returning to the pervious baseline value (from beforethe stimulus), may take much longer. For example, the heart rate of aperson viewing a movie in which there is a startling event may increasedramatically within a second; however, it can take tens of seconds andeven minutes for the person to calm down and for the heart rate returnto a baseline level.

The lag in time it takes affective response to be manifested in certainphysiological and/or behavioral signals can lead to it that the periodin which the affective response is measured occurs after the exposure tothe content. Thus, in some embodiments, measuring the affective responseof the user to the segment of content may end, and possibly even alsostart, essentially after the user is exposed to the segment of content.For example, measuring the user's response to a surprising short scenein a video clip (e.g., a gunshot lasting a second), may involve taking aGSR measurement a couple of seconds after the gunshot was played to theuser. In another example, the user's affective response to playing alevel in a computer game may include taking heart rate measurementslasting even minutes after the game play is completed.

In some embodiments, determining the user's affective response to thesegment of content may utilize measurement values corresponding to afraction of the time the user was exposed to the segment of content. Theuser's affective response to the segment of content may be measured byobtaining values of a physiological signal that is slow to change, suchas skin temperature, and/or slow to return to baseline values, such asheart rate. In such cases, measuring the user's affective response to asegment of content does not have to involve continually measuring theuser throughout the duration in which the user is exposed to the segmentof content. Since such physiological signals are slow to change,reasonably accurate conclusions regarding the user's affective responseto the segment of content may be reached from samples of intermittentmeasurements taken at certain periods during the exposure (the valuescorresponding to times that are not included in the samples can besubstantially extrapolated). In one example, measuring the user'saffective response to playing a computer game involves takingmeasurements during short intervals spaced throughout the user'sexposure, such as taking a GSR measurement lasting two seconds, everyten seconds. In another example measuring the user's response to a videoclip with a GSR, heart rate and/or skin temperature sensor may involveoperating the sensor mostly during certain portions of the video clip,such as a ten-second period towards the end of the clip.

In some embodiments, determining the user's affective response to thesegment of content may involve measuring a physiological and/orbehavioral signal of the user before and/or after the user is exposed tothe segment of content. Optionally, this is done in order to establish abaseline value for the signal to which measurement values of the usertaken during the exposure to the segment of content, and/or shortlyafter the exposure, can be compared. For example, the user's heart ratemay be measured intermittently throughout the duration, of possiblyseveral hours, in which the user plays a multi-player game. The valuesof these measurements are used to determine a baseline value to whichmeasurements taken during a short battle in the game can be compared inorder to compute the user's affective response to the battle. In anotherexample, the user's brainwave activity is measured a few seconds beforedisplaying an exciting video clip, and also while the clip is played tothe user. Both sets of values, the ones measured during the playing ofthe clip and the ones measured before it, are compared in order tocompute the user's affective response to the clip.

Eye Tracking

In some embodiments, “eye tracking” is a process of measuring either thepoint of gaze of the user (where the user is looking) or the motion ofan eye of the user relative to the head of the user. An eye tracker is adevice for measuring eye positions and/or movement of the eyes.Optionally, the eye tracker and/or other systems measure positions ofthe head and/or movement of the head. Optionally, an eye tracker may behead mounted, in which case the eye tracking system measures eye-in-headangles. However, by adding the head position and/or direction toeye-in-head direction, it is possible to determine gaze direction.Optionally, the eye tracker device may be remote relative to the user(e.g., a video camera directed at the user), in which case the eyetracker may measure gaze angles.

Those skilled in the art may realize that there are various types of eyetrackers and/or methods for eye tracking that may be used. In oneexample, eye tracking is done using optical tracking, which track theeye and/or head of the user; e.g., a camera may focus on one or botheyes and record their movement as the user looks at some kind ofstimulus. In another example, eye tracking is done by measuring themovement of an object, such as a contact lens, attached to the eye. Inyet another example, eye tracking may be done by measuring electricpotentials using electrodes placed around the eyes.

In some embodiments, an eye tracker generates eye tracking data bytracking the user 114, for a certain duration. Optionally, eye trackingdata related to a segment of content is generated by tracking the user114 as the user 114 is exposed to the segment. Optionally, the segmentof content includes visual content, and the user 114 is exposed to thesegment by viewing the segment on a display (e.g., on a screen or on ahead-mounted display). Optionally, tracking the user 114 is doneutilizing an eye tracker that is part of a content delivery modulethrough which the user is exposed to content (e.g., a camera embedded ina phone or tablet, or a camera or electrodes embedded in a head-mounteddevice that has a display).

There may be various formats for eye tracking data, and eye trackingdata may provide various insights. For example, eye tracking data mayindicate a direction and/or an object the user 114 was looking at, aduration the user 114 looked at a certain object and/or in certaindirection, and/or a pattern and/or movement of the line of sight of theuser 114. Optionally, the eye tracking data may be a time series,describing for certain points in time a direction and/or object the user114 was looking at. Optionally, the eye tracking data may include alisting, describing total durations and/or time intervals, in which theuser was looking in certain directions and/or looking at certainobjects.

In one embodiment, eye tracking data is utilized to determine agaze-based attention. Optionally, the gaze-based attention is agazed-based attention of the user 114 and is generated from eye trackingdata of the user 114. Optionally, the eye tracking data of the user 114is acquired while the user is consuming content and/or in temporalvicinity of when a user consumes the content. Optionally, gaze-basedattention may refer to a level of attention the user 114 paid to thesegment of content the user 114 consumed.

For example, if the user looks in a direction of the content and focuseson the content while consuming the segment, the gaze-based attentionlevel at that time may be considered high. However, if the user onlyglances cursorily at the content, or generally looks in a directionother than the content while being exposed to the segment, thegaze-based attention level to the segment at that time may be low.Optionally, the gaze-based attention level may be determined for acertain duration, such as a portion of the time a segment of content isdisplayed to the user 114. Thus, for example, different durations thatoccur within the presentation of a certain segment of content may havedifferent corresponding gaze-based attention levels according to eyetracking data collected in each duration.

In one example, a gaze-based attention level of the user to a segment ofcontent may be computed, at least in part, based on difference betweenthe direction of sight of the user, and the direction from the eyes ofthe user to a display on which the segment is presented. Optionally, thegaze-based attention level of the user to a segment of content iscomputed according to the difference between the average direction theuser was looking at during a duration in which the segment was beingdisplayed, compared to the average direction of the display (relative tothe user), during the duration. Optionally, the smaller the differencebetween the direction of sight and the direction of the content, thehigher the gazed-based attention level. Optionally, the gaze-basedattention level may be expressed by a value inversely proportional tothe difference in the two directions (e.g., inversely proportional tothe angular difference).

In another example, a gaze-based attention level of the user 114 to asegment of content may be computed, at least in part, based on theportion time, during a certain duration, in which the user 114 gazes inthe direction of the segment of content (e.g., looking at a module onwhich the content is displayed). Optionally, the gazed-based attentionlevel is proportional to the time spent viewing the content during theduration. For example, if it is determined that the user 114 spent 60%of the duration looking directly at the content, the gaze-basedattention level may be reported as 60%.

In still another example, a gaze-based attention level of the user 114to a segment of content may be computed, at least in part, based on thetime the user 114 spent gazing at certain objects belonging to thecontent. For example, certain objects in the segment may be deemed moreimportant than others (e.g., a lead actor, a product being advertised).In such a case, if the user 114 is determined to be gazing at theimportant objects, it may be considered that the user is payingattention to the content. However, if the user 114 is determined to begazing at the background or at objects that are not important, it may bedetermined that the user 114 is not paying attention to the content(e.g., the user 114 is daydreaming). Optionally, the gaze-basedattention level of the user 114 to the segment of content is a valueindicative of the total time and/or percent of time that the user 114spent during a certain duration gazing at important objects in thesegment of content.

In yet another example, a gaze-based attention level of the user 114 toa segment of content may be computed, at least in part, based on apattern of gaze direction of the user 114 during a certain duration. Forexample, if the user gazes away from the content many times, during theduration, that may indicate that there were distractions that made itdifficult for the user 114 to pay attention to the segment. Thus, thegaze-based attention level of the user 114 to the segment may beinversely proportional to the number of times the user 114 changed thedirection at which the user 114 gazed, e.g., looking and looking awayfrom the content), and/or the frequency at which the user looked awayfrom the content.

In one example, a gaze-based attention level of the user 114 to asegment of content may be computed, at least in part, based onphysiological cues of the eyes of the user. For example, the size of thepupil is known to be linked to the attention level; pupil dilation mayindicate increased attention of the user in the content. In anotherexample, a blinking rate and/or pattern may also be used to determineattention level of the user. In yet another example, if the eyes of theuser are shut for extended periods during the presentation of content,that may indicate a low level of attention (at least to visual content).

In one embodiment, a gaze-based attention level of the user 114 to asegment is computed by providing one or more of the data described inthe aforementioned examples (e.g., values related to direction and/orduration of gaze, pupil size), are provided to a function that computesa value representing the gaze-based attention level. For example, thefunction may be part of a machine learning predictor (e.g., neural net,decision tree, regression model). Optionally, computing the gaze-basedattention level may rely on additional data extracted from sources otherthan eye tracking. In one example, values representing the environmentare used to predict the value, such as the location (at home vs. in thestreet), the number of people in the room with the user (if alone it iseasier to pay attention than when with company), and/or thephysiological condition of the user (if the user is tired or drunk it ismore difficult to pay attention). In another example, values derivedfrom the content may be used in computing the attention level, such asthe type or genre of content, the duration of the content, may also befactors that may be considered in the computation. In yet anotherexample, prior attention levels of the user and/or other users tosimilar content may be used in the computation (e.g., a part that manyusers found distracting may also be distracting to the user).

In one embodiment, a gaze-based attention level is represented by one ormore values. For example, the attention level may be a value between 1and 10, with 10 representing the highest attention level. In anotherexample, the attention level may be a value representing the percentageof time the user was looking at the content. In yet another example, theattention level may be expressed as a class or category (e.g., “lowattention”/“medium attention”/“high attention”, or “looking atcontent”/“looking away”). Optionally, a classifier (e.g., decision tree,neural network, Naive Bayes) may be used to classify eye tracking data,and possibly data from additional sources, into a class representing thegaze-based attention level.

Content

Content the user consumes during interactions with a digital device cantake many forms. Optionally, a user consumes content by being exposed tothe content and/or from gaining and/or learning information due to theexposure to the content. The content may also originate from varioussources. One thing common to the various forms of content is thatcontent conveys information, and by being exposed to the content, theuser may gain and/or learn certain information. Additionally, in somecases, gaining the information conveyed by the content may cause theuser to have an affective response; such an affective response isconsidered an affective response of the user to the content (or phrasedalternatively, “the user's affective response to the content”).

In one embodiment, the user is exposed to a segment of content that isgenerated as part of a conversation between entities, such as humansand/or computers (e.g., an artificial intelligence). The conversationmay be between same types of entities (e.g., a conversation betweenhumans), or different types of entities (e.g., a conversation between auser and a computer). Optionally, one of the sides to the conversationmay be the user 114. The conversation may take place utilizing one ormore channels for conveying information, such as a voice conversation,video conversation, exchange of instant messages, and dialogue betweencharacters in a virtual environment (e.g., characters in a multi-playergame).

In one embodiment, the user is exposed to content comprising a textmessage. The text message may be one or more of the following: aninstant message, an e-mail, blog post, an article, and/or status updateon a social network site.

In one embodiment, the user is exposed to content comprising images suchas still images and/or video. In one example, the content originatesfrom a computerized source, such as a video clip from an Internet site,and/or a video content generated by a computer game (e.g., depictingcharacters in a virtual world).

In one embodiment, the user is exposed to content comprising non-verbalaudio sounds. In one example, the content includes music. In anotherexample, the content includes sound effects, such as sound effectsaccompanying a computer game (e.g., noises corresponding to shots and/orexplosions).

In one embodiment, the user is exposed to content in the form ofmessages and/or feedback from a computerized system the user interactswith. For example, these messages can be system messages such as anindication that the user received a message, an event reminder, ahardware-related message (e.g., no wireless signal). In another example,the content may include feedback from a message composition system, suchas highlighting spelling and/or grammar mistakes in a text beingcomposed by a user.

In some embodiments, the segment of content includes one or moreportions of content that the user may be exposed to over a period oftime. In some cases, a portion of content may belong to multiplesegments, for example, a scene from a movie may belong to severalsegments of the movie, having different lengths and/or starting times inthe movie.

The segment of content may possess varying lengths and sizes, optionallydepending on the type of content the segment includes and the context inwhich the segment is used. In one example, the segment of contentincludes a portion of video lasting a fraction of a second (e.g., aportion of video depicting a gunshot). In another example, the segmentof content involves a whole movie, and in this case the segment may spanhours.

In some embodiments, the segment of content may involve a single item towhich the user may be exposed. For example, the segment of content maybe a message, a sentence, a conversation, a video clip, a videogenerated while playing a level in a computer game, or a whole movie. Inother cases, the segment of content may be a portion of a content item.For example, the segment of content may include part of an utteredsentence, a few seconds from a video clip, video comprising a few playmoves in a computer game, or a single correction suggested by a systemin text composition. And in yet other cases, the segment of content mayinclude multiple content items. For example, the segment of content mayinclude multiple commercials the user views, an email or text messagethread, and/or computer game content from playing multiple games.

In some embodiments, the segment of content may include multiple items,and the user may be concurrently exposed to more than one of the itemsincluded in the segment of content. For example, a user viewing contenton a social network site may be exposed simultaneously to several statusposts, images, and/or videos, which together can be considered a segmentof social network content, or multiple segments of content, optionallydepending on the context in which the content is examined.

Predictor

In one embodiment, a module that receives a query that includes a sample(e.g., a vector of feature values), and predicts a label for that sample(e.g., a class associated with the sample), is referred to as a“predictor”. A sample provided to a predictor in order to receive aprediction for it may be referred to as a “query sample”. Additionally,the pair that includes a sample and its corresponding label may bereferred to as a “labeled sample”.

In some embodiments, a sample for a predictor (e.g., a sample used astraining data and/or a query sample) includes one or more featurevalues. Optionally, at least some of the feature values are numericalvalues. Optionally, at least some of the feature values may becategorical values that may be represented as numerical values (e.g.,via indexes for different categories).

In some embodiments, a label that may serve as prediction value for aquery sample provided to a predictor, may take one or more types ofvalues. For example, a label maybe include a discrete categorical value(e.g., a category), a numerical value (e.g., a real number), and/or amultidimensional value (e.g., a point in multidimensional space).

In one embodiment, a predictor utilizes a model in order to makepredictions for a given query sample. There is a plethora of machinelearning algorithms for training different types of models that can beused for this purpose. Some of the algorithmic approaches that may beused for creating the predictor are classification, clustering, functionprediction, and/or density estimation. Those skilled in the art canselect the appropriate type of model depending on the characteristics ofthe training data (e.g., its dimensionality), and/or the type of valueused as labels (e.g., discrete value, real value, or multidimensional).

For example, classification methods like Support Vector Machines (SVMs),Naive Bayes, nearest neighbor, and/or neural networks can be used tocreate a predictor of a discrete class label. In another example,algorithms like a support vector machine for regression, neuralnetworks, and/or gradient boosted decision trees can be used to create apredictor for real-valued labels, and/or multidimensional labels. In yetanother example, a predictor may utilize clustering of training samplesin order to partition a sample space such that new query samples can beplaced in clusters and assigned labels according to the clusters theybelong to. In somewhat similar approach, a predictor may utilize acollection of labeled samples in order to perform nearest neighborclassification (in which a query sample is assigned a label according tothe labeled samples that are nearest to them in some space).

In one embodiment, semi-supervised learning methods are used to train apredictor's model, such as bootstrapping, mixture models and ExpectationMaximization, and/or co-training. Semi-supervised learning methods areable to utilize as training data unlabeled samples in addition to thelabeled samples.

In one embodiment, a predictor may return as a label other samples thatare similar to a given query sample. For example, a nearest neighborapproach method may return one or more samples that are closest in thedata space to the query sample (and thus in a sense are most similar toit.)

In one embodiment, in addition to a label predicted for a query sample,a predictor may provide a value describing a level of confidence in itsprediction of the label. In some cases, the value describing theconfidence level may be derived directly from the prediction processitself. For example, a predictor utilizing a classifier to select alabel for a given query sample may provide a probability or scoreaccording to which the specific label was chosen (e.g., a Naive Bayes'posterior probability of the selected label, or a probability derivedfrom the distance of the sample from the hyperplane when using an SVM).

In one embodiment, a predictor making a prediction for a query samplereturns a confidence interval as its prediction or in addition to apredicted label. A confidence interval is a range of values and anassociated probability that represents the chance that the true valuecorresponding to the prediction falls within the range of values. Forexample, if a prediction is made according to an empirically determinedNormal distribution with a mean m and variance ŝ2, the range [m−2s,m+2s]corresponds approximately to a 95% confidence interval surrounding themean value m.

The type and quantity of training data used to train a predictor's modelcan have a dramatic influence on the quality of the predictions made bythe predictor. Generally speaking, the more data available for traininga model, and the more the training samples are similar to the samples onwhich the predictor will be used (also referred to as test samples), themore accurate the predictions for the test samples are likely to be.Therefore, when training a model that will be used to make predictionsregarding a specific user, it may be beneficial to collect training datafrom the user (e.g., data comprising measurements of the specific user).

Emotional Response Predictors (ERP)

In the embodiments, a predictor that predicts a label that is related toan emotional response may be referred to as a “predictor of emotionalresponse” or an Emotional Response Predictor (ERP). A predictor ofemotional response that receives a query sample that includes featuresthat describe a segment of content may be referred to as a predictor ofemotional response from content, a “content emotional responsepredictor”, and/or a “content ERP”. Similarly, a predictor of emotionalresponse that receives a query sample that includes features derivedfrom measurements of a user, such as affective response measurementstaken with a device and/or a sensor, may be referred to as a predictorof emotional response from measurements, a “measurement emotionalresponse predictor”, and/or a “measurement ERP”. Additionally, a modelutilized by an ERP to make predictions may be referred to as an“emotional response model”.

In some embodiments, a model used by an ERP (e.g., a content ERP and/ora measurement ERP), is primarily trained on data collected from one ormore different users that are not the user 114; for instance, at least50% of the training data used to train the model does not involve theuser 114. In such a case, a prediction of emotional response madeutilizing such a model may be considered a prediction of the emotionalresponse of a representative user. It is to be noted that therepresentative user may in fact not correspond to an actual single user,but rather correspond to an “average” of a plurality of users.Additionally, under the assumption that the user 114 has emotionalresponses that are somewhat similar to other users' emotional responses,the prediction of emotional response for the representative user may beused in order to determine the likely emotional response of the user114.

In some embodiments, a label returned by an ERP may represent anaffective response, such as a value of a physiological signal (e.g.,GSR, heart rate) and/or a behavioral cue (e.g., smile, frown, blush).

In some embodiments, a label returned by an ERP may be a valuerepresenting a type of emotional response and/or derived from anemotional response. For example, the label my indicate a level ofinterest and/or whether the response can be classified as positive ornegative (e.g., “like” or “dislike”).

In some embodiments, a label returned by an ERP may be a valuerepresenting an emotion. In the embodiments, there are several ways torepresent emotions (which may be used to represent emotional states andemotional responses as well). Optionally, but not necessarily, an ERPutilizes one or more of the following formats for representing emotionsreturned as its predictions.

In one embodiment, emotions are represented using discrete categories.For example, the categories may include three emotional states:negatively excited, positively excited, and neutral. In another example,the categories include emotions such as happiness, surprise, anger,fear, disgust, and sadness.

In one embodiment, emotions are represented using a multidimensionalrepresentation, which typically characterizes the emotion in terms of asmall number of dimensions. In one example, emotional states arerepresented as points in a two dimensional space of Arousal and Valence.Arousal describes the physical activation and valence the pleasantnessor hedonic value. Each detectable experienced emotion is assumed to fallin a specified region in that 2D space. Other dimensions that aretypically used to represent emotions include: potency/control (refers tothe individual's sense of power or control over the eliciting event),expectation (the degree of anticipating or being taken unaware), andintensity (how far a person is away from a state of pure, coolrationality). The various dimensions used to represent emotions areoften correlated. For example, the values of arousal and valence areoften correlated, with very few emotional displays being recorded withhigh arousal and neutral valence. In one example, emotions arerepresented as points on a circle in a two dimensional space pleasureand arousal, such as the circumflex of emotions.

In one embodiment, emotions are represented using a numerical value thatrepresents the intensity of the emotional state with respect to aspecific emotion. For example, a numerical value stating how much theuser is enthusiastic, interested, and/or happy. Optionally, the numericvalue for the emotional state may be derived from a multidimensionalspace representation of emotion; for instance, by projecting themultidimensional representation of emotion to the nearest point on aline in the multidimensional space.

In one embodiment, emotional states are modeled using componentialmodels that are based on the appraisal theory, as described by the OCCmodel (Ortony, A.; Clore, G. L.; and Collins, A. 1988. The CognitiveStructure of Emotions. Cambridge University Press). According to thistheory, a person's emotions are derived by appraising the currentsituation (including events, agents, and objects) with respect to theperson goals and preferences.

In one embodiment, a measurement ERP is used to predict an emotionalresponse of a user from a query sample that includes feature valuesderived from affective response measurements. Optionally, the affectiveresponse measurements are preprocessed and/or undergo feature extractionprior to being received by the measurement ERP. Optionally, theprediction of emotional response made by the measurement ERP is aprediction of the emotional response of the user 114. Alternatively oradditionally, the prediction of emotional response made by themeasurement ERP is a prediction of emotional response of arepresentative user.

There are various methods in which a measurement ERP may predictemotional response from measurements of affective response. Examples ofmethods that may be used in some embodiments include: (i)physiological-based predictors as described in Table 2 in van den Broek,E. L., et al. (2010) Prerequisites for Affective Signal Processing(ASP)—Part II. In: Third International Conference on Bio-InspiredSystems and Signal Processing, Biosignals 2010; and/or (ii) Audio- andvisual-based predictors as described in Tables 2-4 in Zeng, Z., Pantic,M., Roisman, G. I., and Huang, T. S. (2009) A Survey of AffectRecognition Methods: Audio, Visual, and Spontaneous Expressions. IEEETransaction on Pattern Analysis and Machine Intelligence, Vol. 31(1),39-58.

In one embodiment, a measurement ERP may need to make decisions based onmeasurement data from multiple types of sensors (often referred to inthe literature as multiple modalities). This typically involves fusionof measurement data from the multiple modalities. Different types ofdata fusion may be employed, for example feature-level fusion,decision-level fusion, or model-level fusion, as discussed in Nicolaou,M. A., Gunes, H., & Pantic, M. (2011), Continuous Prediction ofSpontaneous Affect from Multiple Cues and Modalities in Valence-ArousalSpace, IEEE Transactions on Affective Computing.

In one embodiment, a content ERP is used to predict an emotionalresponse of a user from a query sample that includes feature valuesderived from a segment of content. Optionally, the segment of content ispreprocessed and/or undergoes feature extraction prior to being receivedby the content ERP. Optionally, the prediction of emotional response tothe segment of content made by the content ERP is a prediction of theemotional response of the user 114 to the segment of content.Alternatively or additionally, the prediction of emotional response tothe segment of content made by the content ERP is a prediction ofemotional response of a representative user.

In one embodiment, feature values are used to represent at least someaspects of a segment of content. Various methods may be utilized torepresent aspects of a segment of content into feature values. Forexample, the text in a segment that includes text content can beconverted to N-gram or bag of words representations, in order to set thevalues of at least some of the feature values. In another example, animage or video clip from a segment that includes visual content may beconverted to features by applying various low-pass and/or high-passfilters; object, gesture and/or face recognition procedures; genrerecognition; and/or dimension reduction techniques. In yet anotherexample, auditory signals are converted to features values such aslow-level features describing acoustic characteristics such as loudness,pitch period and/or bandwidth of the audio signal. In still anotherexample, semantic analysis may be utilized in order to determine featurevalues that represent the meaning of the content of a segment.

There are many feature extraction methods mentioned in the literaturethat can be utilized to create features for audio-, image-, and/orvideo-containing content. For example, useful feature extraction methodsare used in areas such as visual content-based video indexing andretrieval, automatic video highlighting, and affective video contentrepresentation and modeling.

In one embodiment, training data used to create a content ERP iscollected from one or more users. Optionally, a sample used as trainingdata is derived from a segment of content to which a user is exposed;the sample's corresponding label may be generated from measurements ofthe user's affective response to the segment of content, e.g., byproviding the measurements to a measurement ERP. Optionally, at least aportion of the training data is collected from the user 114.Additionally or alternatively, at least a portion of the training datais collected from a set of users that does not include the user 114.

In one embodiment, to make its predictions, the content ERP utilizesfeature values that describe aspects beyond the scope of the dataconveyed in the content. These additional aspects can have an impact ona user's emotional response to the content, so utilizing feature valuesdescribing these values can make predictions more accurate.

In particular, in many cases, a prediction of a user's emotionalresponse to content may depend on the context and situation in which thecontent is consumed. For example, for content such as an action movie, auser's emotional response might be different when viewing a movie withfriends compared to when viewing alone (e.g., the user might be moreanimated and expressive with his emotional response when viewing withcompany). However, the same user's response might change dramatically touneasiness and/or even anger if younger children are suddenly exposed tothe same type of content in the user's company. Thus, context andsituation, such as who is consuming content with the user can have adramatic effect on a user's emotional response.

Similarly, a user's emotional state, such as a user's mood, can alsoinfluence a user's emotional response to content. For example, whileunder normal circumstances, a slapstick oriented bit of comedy might bedismissed by a user as juvenile, a user feeling depressed might actuallyenjoy it substantially more (as a form of a comedic “pick-me-up”), andeven laugh heartily at the displayed comedic antics.

Therefore, in order to capture information regarding context and/orsituation in which a user consumes the content, in some embodiments,samples that may be provided to an ERP include feature values describingthe context in which the content is consumed and/or the user'ssituation. For example, these feature values may describe aspectsrelated to the user's location, device on which the content is consumed,people in the user's vicinity, tasks or activities the user performed orneeds to perform (e.g., work remaining to do), and/or the user's orother peoples emotional state as determined, for example, from analyzingcommunications of a log of the activities of the user and/or otherpeople related to the user. In another example, the feature valuesdescribing context and/or situation may include physiologicalmeasurements and/or baseline values (e.g., current and/or typical heartrate) of the user and/or other people.

As well as consuming content, a user interacting with a digital devicemay also generate content that can undergo analysis. For example,messages created by a user (e.g., a spoken sentence and/or a textmessage), are user-generate content that may be analyzed to determinethe user's emotional state (e.g., using voice stress analysis, and/orsemantic analysis of a text message). In another example, informationregarding the way a user plays a game, such as the number of times theuser shoots in a shooter game and/or the type of maneuvers a userperforms in a game that involves driving a vehicle, are alsouser-generated content that can be analyzed. Therefore, in oneembodiment, one or more features derived from a segment ofuser-generated content are included in a sample for the content ERP, inorder to provide further information on context in which the content isconsumed and/or on the user's situation.

One source of data that has been found useful for predicting a user'semotional response to content has been the emotional responses of otherusers to content (an approach sometimes referred to as “collaborativefiltering”). In one embodiment, a content ERP utilizes data regardingother users' emotional responses to content. For example, by comparing auser's emotional response to certain segments of content, with theemotional responses other users had to at least some of those segments,it is possible to find other users that respond similarly to the user inquestion. These users may be said to have a similar response profiles tothe user. Thus, in order to predict the user's response to a previouslyunobserved segment of content, a content ERP may rely on the responsesthat other users, with a similar response profiles to the user, had tothe unobserved segment.

Content ERA

In some embodiments, the content ERA (Emotional Response Analyzer) isused to analyze a segment of content. Optionally, the content ERAincludes and/or utilizes a module that evaluates the segment of contentto determine to what extent a user's affective response to the segmentof content should be measured and/or to what extent measurements of theuser's affective response to the segment of content should be processed.Additionally, the content ERA may generate an indication which isforwarded to the controller, and the controller may select a mode ofoperation for a device with respect to the indication.

In one embodiment, the content ERA includes a content ERP and/orutilizes a content ERP. In one example, the content ERA uses the contentERP to predict an emotional response to the content, such as whether theuser will be interested in the content and/or to what degree the contentis likely to make the user feel one or more feelings such as happiness,content, anxiousness, fear. In yet another example, the content ERA usesa predictor to predict a user's affective response to content, such asto what extent the user's heart-rate is expected to rise due to beingexposed to the content, and/or whether the user expected to smile due tobeing exposed to the content.

In one embodiment, the content ERA evaluates whether determining auser's emotional response to the segment of content will be usefuland/or beneficial. In this embodiment, determining the user's emotionalresponse to the segment of content may involve measuring the user'saffective response with a device and/or processing affective responsemeasurements of the user. Based on its analysis, the content ERA maygenerate an indication that indicates to the controller whether it isworthwhile to select a mode of operation in which the user issubstantially measured.

In one embodiment, the content ERA utilizes data related to the contentand/or describing the content in its analysis of the segment of content.The content ERA may receive the data in addition to, or instead of, thesegment of content. For example, the content ERA may receive informationregarding the popularity of the content with certain demographics,information regarding the emotional response to the content by certainpeople or subsets of people, and/or information regarding the priceand/or circumstances under which the content may be consumed.Optionally, at least some of the data received by the content ERA isprovided to a measurement ERP in order to add context that may help themeasurement ERP make more accurate predictions.

In some embodiments, the content ERA checks whether a value related to aprediction of emotional response to the segment of content is expectedto reach a predetermined threshold. The content ERA may output anindication that is forwarded to the controller which is indicative ofthe whether or not the value, related to the prediction of the emotionalresponse, reaches the predetermined threshold. Additionally, theindication may include information regarding the difference between thevalue and the predetermined threshold. Alternatively, the content ERAmay elect not to send an indication to the controller if the valuerelated to the emotional response to the segment of content does notreach the predetermined threshold.

In one embodiment, the predetermined threshold utilized by the contentERA includes a value representing a predicted emotional response of auser to the segment of content. Optionally, the emotional response maybe expressed in terms of values of affective response. Given aprediction from a content ERP utilized by the content ERA, the contentERA compares the prediction to the predetermined threshold, anddetermines whether the predetermined threshold is reached. Optionally,the prediction of emotional response to the segment of content, which iscompared to the predetermined threshold, is a prediction of theemotional response of the user 114 to the segment of content.Alternatively or additionally, the prediction of emotional response tothe segment of content that is compared may be a prediction of emotionalresponse of a representative user.

The predetermined threshold may represent an absolute level of emotionalresponse that is expected to be expressed by the user due to beingexposed to the segment of content and/or an expected level of change tothe user's emotional response due to the exposure to the segment.

In one example, the content ERP is used to predict an increase in theuser's heart rate in order to determine if the user is excited. In thiscase, a predetermined threshold of an (expected) increase of 10 beatsper minute may be used to determine if exposure to content causes theuser to be excited. Thus, if the content ERP predicts a change to theuser's affective response that includes an expected increase of 20 beatsper minute to the user's heart rate, this means that the prediction hasreached the predetermined threshold. The indication that the content ERAoutputs in this example may indicate to the controller to select a modeof operation that is appropriate for measuring the user with a devicewhen the user is expected to have a noticeable affective response.

In another example, the content ERA may utilize a predictor to determinehow long a segment of content is likely to hold a user's interest. Thecontent ERA may utilize a threshold of a minimum duration of interestthat depends on the type of content (e.g., 20 seconds for a video, 10seconds for a short post on a social network). Thus, if a prediction ofemotional response to content includes a duration in which the contentis expected to hold the user's attention, the duration can be comparedto the predetermined threshold in order to determine whether thepredetermined threshold is reached. The indication that the content ERAmay output in this example may indicate to the controller to select amode of operation that is appropriate to the duration of predictedinterest in the content (e.g., if the duration is below the threshold,there is no need to measure the user).

In one embodiment, the predetermined threshold utilized by the contentERA includes a value representing a threshold of minimal benefit thatmight be gained from determining a user's emotional response to thesegment of content. Given the segment of content, for which a user'semotional response may be predicted and/or measured, the content ERAestimates how beneficial it would be to obtain substantiated informationregarding the user's emotional response to the content. Optionally, thevalue representing the benefit that might be obtained from determiningthe emotional response to the segment of content refers to the benefitof determining the emotional response of the user 114 to the segment ofcontent. Alternatively or additionally, the value representing thebenefit may refer to the benefit of determining the emotional responseof a representative user. The content ERA can then compare the estimatedbenefit to the predetermined threshold in order to determine whether thethreshold is reached, and output an indication that is indicative ofwhether the benefit reaches the threshold. This indication may beforwarded to the controller that may use it in order to select anappropriate mode of operation for a device.

In one example, the content ERA utilizes a procedure that determines thetype of content the user is to be exposed to, such as a procedure thatdetermines the genre of video segments. The content ERA also has accessto information regarding previous affective response measurements of theuser, obtained for the purpose of training an emotional response model,while the user viewed various genres of content. In this example, thesystem needs to collect additional training data comprising the user'saffective responses, in order to train a more personalized model for theuser. The content ERA may determine that in some cases, the benefit ofcollecting additional measurements is likely to be low; such as whenmeasuring the user's response to a segment of content belonging to genrefor which there is already ample training data. In other cases, thecontent ERA might determine that the benefit to be obtained fromadditional measurements of the user, is likely to be high; such as whenthe user is viewing a segment of content from a genre for which thereare few samples of the user's response. Thus, the content ERA canutilize a benefit threshold derived from the amount of previous datacollected of a user's affective response to content of different genres,in order to determine whether a given segment of content that belongs toa certain genre, reaches the threshold. The content ERA can output anindication that is indicative of whether or not the benefit frommeasuring the user's affective response to the segment of contentreaches the threshold. The controller may utilize the indication inorder to select an appropriate mode of operation for a device, withrespect to the benefit indicated in by the indication.

In another example, a user is monitored while consuming content in orderto improve the user's experience (e.g., by modifying elements in thecontent such as rendering a personalized video). In such a case, thecontent ERA may be used to determine details regarding the content, suchas the type of content, and whether the user experience can be enhancedwith the type of content being consumed by the user. If the content ERAdetermines that the content cannot be enhanced (e.g., it does notcontain elements for which there are suitable personalization services),the content ERA may determine that the content it analyzed does notwarrant measuring of the user's affective response since there isnothing beneficial to do with the affective response measurement data.The indication generated by the content ERA in this case may indicate tothe controller to select a mode of operation in which the user issubstantially not measured.

Semantic Analysis

Semantic analysis is often used to determine the meaning of content fromits syntactic structure. In some embodiments, semantic analysis may beused by the content ERA in its analysis of a segment of content.Optionally, semantic analysis of the segment of content may be used tocreate feature values that represent the meaning of a portion of thesegment of content; such as features describing the meaning of one ormore words, one or more sentences, and/or the full segment of content.

Providing insight into the meaning of the segment of content may help topredict the user's emotional response to the segment of content moreaccurately. For example, a segment of content that is identified asbeing about a subject that the user likes, is likely to cause the userto be interested and/or evoke a positive emotional response. In anotherexample, being able to determine that the user received a message thatexpressed anger (e.g., admonition of the user), can help to reach theconclusion that the user is likely to have a negative emotional responseto the content. Optionally, the semantic analysis is utilized in theprediction of emotional response of the user 114 to the segment ofcontent. Alternatively or additionally, the semantic analysis isutilized in the prediction of emotional response of a representativeuser to the segment of content.

In one embodiment, the content ERA utilizes semantic analysis in orderto create an indication that is forwarded to the controller. Theindication may indicate to the controller to select an appropriate modeof operation for a device when measuring the user's affective responseto the segment of content. For example, semantic analysis may determinethat a segment of content includes statements of affection that the useris likely to appreciate, so the content ERA produces an indication thatindicates to the controller to select a mode of operation in which theuser's affective response to the content is measured by a device (asopposed to a mode of operation in which the device remains in a standbymode). In another example, semantic analysis may determine that thesegment of content involves a subject that does not interest the user,so the content ERA produces an indication that indicates to thecontroller to select a mode of operation in which the user is notsubstantially measured (since there is not likely to be an interestingemotional response to the content).

Semantic analysis of content can utilize various procedures that providean indication of the meaning of the content.

In one embodiment, Latent Semantic Indexing (LSI) and/or Latent SemanticAnalysis (LSA) are used to determine the meaning of content comprisingtext (e.g., a paragraph, a sentence, a search query). LSI and LSAinvolve statistically analyzing the frequency of words and/or phrases inthe text in order to associate the text with certain likely conceptsand/or categories. The information regarding the concept and/or categoryof the segment of content can be utilized by the content ERA to generatean indication that is indicative of the extent at which the user'saffective response should be measured. For example, the concepts orcategories describing the text, which were determined using LSI may beencoded as feature values that are provided to an emotional responsepredictor. The content ERA can then use a prediction of the emotionalresponse to produce an indication that indicates the controller toselect an appropriate mode of operation for the device while it is usedto measure the user's affective response to the segment of content.

In one example, the content ERA uses LSI in order to assign segments ofa text with a label describing the subject of the text. The LSI involvesa model that is trained on samples comprising a plurality of texts withtheir corresponding subjects. By comparing the frequencies of words andphrases in a new text with the frequencies of words and phrases in textscorresponding to different subjects, it is possible to determine alikely subject for the new text. Thus, using the LSI model the contentERA can detect if a segment of content is likely or not to involve asubject that interests the user, and generate an indication thatindicates the controller to select an appropriate mode of operation forthe device based on its determination of the user's interest in thesubject of the segment (e.g., if the subject interests the user, theuser should be measured extensively, and if it doesn't, the user shouldbe measured rudimentarily).

In one embodiment, semantic analysis of a segment of content utilizes alexicon that associates words and/or phrases with their core emotions.For example, the analysis may utilize a lexicon similar to the onedescribed in “The Deep Lexical Semantics of Emotions” by Hobbs, J. R.and Gordon, A. S., appearing in Affective Computing and SentimentAnalysis Text, Speech and Language Technology, 2011, Volume 45, 27-34,which describe the manual creation of a lexicon that classifies wordsinto 32 categories related to emotions. The information regarding thecore emotions of words can be utilized by the content ERA in order todetermine the user's likely emotional response to a segment of content.For example, the core emotion associated with words belonging to thesegment of content can be provided as feature values to a predictor ofemotional response to content. The content ERA can then utilize theprediction to create an indication that is indicative of the extent atwhich affective response of the user should be measured by a device.

In one embodiment, semantic analysis of content involves using analgorithm for determining emotion expressed in text. The information onthe emotion expressed in the text may be used in order to provideanalysis algorithms with additional semantic context regarding theemotional narrative conveyed by text. For example, algorithms such asthe ones described in “Emotions from text: machine learning fortext-based emotion prediction” by Alm, C. O. et al, in the Proceedingsof Human Language Technology Conference and Conference on EmpiricalMethods in Natural Language (2005), pages 579-586, can be used toclassify text into the basic emotions such as anger, disgust, fear,happiness, sadness, and/or surprise. The information on the emotionexpressed in the text can be provided as feature values to a predictorof emotional response that is utilized by the content ERA in order toproduce an indication that is indicative of the extent at which the usershould be measured. The controller receives the indication and can thenselect an appropriate mode of operation for the device with respect tothe indication.

In one example, the content ERA uses a predictor of emotional responseto predict the user's emotional response to a received message thatincludes text, or a message that can be converted to text. The predictorreceives feature values that include both the expressed emotions in thetext (as determined by semantic analysis), and optionally, contextualfeatures corresponding to the message, such as the identity of thesender of the message, the time the message was sent, and/or thelocation of the user upon receiving the message. The content ERAproduces an indication according to the prediction of emotional responsethat is indicative of the extent at which the user's affective responseshould be measured. Optionally, the prediction of emotional responseutilized by the content ERA is a prediction of the emotional response ofthe user 114 to the content. Alternatively or additionally, theprediction of emotional response may be a prediction of emotionalresponse of a representative user.

In one embodiment, a segment of content to which the user is exposedincludes information that can be converted to text. For example, vocalcontent such as a dialogue is converted to text using speech recognitionalgorithms, which translate spoken text into words. Optionally, the textof the converted content is subjected to semantic analysis methods.

Controller

In some embodiments, a controller, such as the controller 108, selects amode of operation for a device used to measure affective response basedon an indication provided by a content ERA. Optionally, the controllerselects a mode of operation from amongst a plurality of modes ofoperation, where at least some of the modes of operation differ withrespect to the power the device is configured to consume when operatingin those particular modes of operation.

In one example, the content ERA provides an indication that isindicative of whether a predetermined threshold was reached by a valueobtained from analyzing a segment of content. Depending on whether thethreshold was reached, and/or the difference between the value and thethreshold, the controller may select a mode of operation from a setcomprising a plurality of different modes of operation, with possiblydifferent power consumption rates.

In another example, the content ERA provides an indication that includesa value that is indicative of a level of importance that may beattributed to the need to measure the user's affective response to thecontent. For instance, the indication may include a predicted interestlevel of a user in content, and the controller selects a mode ofoperation, such as a resolution of images captured by a camera,according to the interest level. In this example, when the user isexpected to be interested in content, it is important to take detailedmeasurement of the user's affective response; so for instance, the modeof operation of a camera used to capture images of the user could be setto a higher resolution when a user is expected to be interested.

In some embodiments, selecting a certain mode of operation for a device,such as the device 112, may cause the device to consume less power,compared to the power it consumes in one or more other modes ofoperation. There are different possibilities in which a certain mode ofoperation can cause the device to operate in a way that reduces itspower consumption:

In one embodiment, the device can consume significantly less power bybeing in a mode in which it is substantially turned off (or one or moreof its components are essentially turned off), and thus substantiallydoes not consume power at all. Alternatively, the device may be put in astandby mode of operation. Optionally, when operating in the standbymode, the device does not consume the same power the way it consumes ina regular mode of operation, but rather it consumes significantly less,but sufficient to enable the device to return to normal operation whenneeded (e.g., upon receiving a signal to do so).

In another embodiment, the device may consume significantly less powerby operating in a mode of operation in which measurement data producedby the device is of a lower quality, or in a mode of operation in whichless measurement data is collected by the device, per unit ofmeasurement time.

In one example, a device that is an image capturing device can consumesignificantly less power by capturing images at a lower resolution.Similarly, if the image capturing device is utilized to capture video, asignificantly lower power consumption can be achieved by reducing theframe rate at which the video is captured.

In another example, a device such as a physiological sensor (e.g., anEEG sensor or a GSR sensor), can be configured to draw less power (e.g.,by operating at a lower voltage). This can cause the measurements takenby the sensor to be noisier, e.g., they may contain more randomfluctuations and artifacts.

Some devices contain components that perform processing of the dataprior to it being sent to other elements. In order to conserve power, insome embodiments, the device may perform less processing operations ondata it measures. For example, an image capturing device may havehardware components that are able to compress images and/or video priorto saving the file in a compressed format (such as jpeg, gif, avi). Inorder to conserve power, in a certain mode of operation, the image maybe compressed using a simple lossy compression algorithm whichsignificantly reduces the image quality (but requires relatively fewcomputations to run).

In some embodiments, a device may include a component enabling it totransmit the data it collects, e.g., via wireless communication. Inorder to conserve power, in a certain mode of operation, the device maybe configured not to transmit some or all of the data it collects in acertain time period. Power may also be conserved in a certain mode ofoperation by configuring the device to transmit data utilizing a weakersignal (which consumes a lower wattage but provides less protectionagainst noise).

In one embodiment, when a controller, such as the controller 108,selects a mode of operation, it takes into consideration additionalinformation that might not be conveyed in an indication received from acontent ERA and/or a semantic analyzer. For example, the controller of adevice that is battery operated sensor might take into consideration thecurrent charge in the battery used by the sensor, when choosing a modeof operation to operate in.

In one embodiment, a controller, such as the controller 154 or thecontroller 204, is used to select a processing level for a processorthat processes measurements of affective response. Being able to selectdifferent processing levels enables the system to reduce thecomputational load involved in processing measurements of affectiveresponse that are collected over time. The reduction in the computationload is achieved by causing the processor to operate in a mode thatutilizes fewer computation cycles, per volume unit of measurement data,at least with measurement data collected during some of the periodsduring which a user is measured.

In some embodiments, a processing level for a processor is selected froma plurality of processing levels, where at least some of the processinglevels differ with respect to the number of computation cycles theyutilize per volume unit of measurement data available for processing(e.g., per kilobyte of data).

In one example, an indication that indicates whether a predeterminedthreshold was reached by a value related to a prediction of emotionalresponse to a segment of content. Depending on whether the threshold wasreached, and/or the difference between the value and the threshold, aspecific processing level may be selected from amongst multipledifferent processing levels, possibly having different processingprocedures invoked to process the data and/or with different operatingparameters for the procedures invoked to process the data.

In another example, an indication, according to which a processing levelis selected, may include a value that is indicative of a level ofimportance that may be attributed to the need to measure and/or processthe user's affective response to a segment of content. For instance, theindication may include a predicted interest level of a user in content.Based on the value, a certain processing level may be selected, whichdetermines what type of feature extraction is to be performed onaffective response measurements that include images of a user's faceacquired with a camera (the images record the user's reaction whenviewing the segment of content). In this example, when the user isexpected to be interested in the content, images may be processed moreextensively and in order to detect facial expressions andmicro-expressions.

Processing levels may differ in the computational load they impose onthe processor for processing data, i.e., different processing levels mayutilize, on average, different amounts of computation cycles to processa volume unit of measurement data that is available for processing.There may be different possibilities in which a processing level cancause a processor, such as the processor 156 or the processor 206, tooperate in a way that reduces the computational load:

In one embodiment, the processor can reduce its computational load bynot processing the available data (e.g., the processor is idle).Additionally, a certain processing level may indicate to a processor notto process affective response measurement data. For example, theprocessor might not access the data, or not retrieve it from memorystorage.

In another embodiment, the processor may reduce its computational loadby processing only a portion of the data at its disposal. For example, acertain processing level may indicate to the processor to sample andprocess 10% of the available measurement data (e.g., process every tenthsecond), even though the other 90% is available for it to process. Inthis case, the processor may utilize significantly fewer computationalcycles to process the data that was provided to it, compared to aprocessing level in which it processes all the data provided to it.

Processing affective response measurement data may involve runningvarious procedures on the data, such as performing various analyses. Inone embodiment, the processor may be configured to be able to run a setof procedures that includes a plurality of data processing procedures,and the processing level chosen by the controller indicates to theprocessor which of the procedures to run on the data. For example, theprocessor may be configured to process affective response measurementdata that includes physiological measurements. One of the procedures theprocessor may run on the data performs simple averaging of values (e.g.,in order to report a baseline value), while another procedure my involvenormalizing, filtering, and performing signal analysis on the data inorder to submit the data to further analysis such as emotional responseprediction. Thus, a first processing level that indicates to theprocessor to perform the former procedure is likely to cause theprocessor to utilize, per volume unit of data processed, a small numberof computation cycles. In contrast, a second processing level thatindicates to the processor to perform the latter procedure is likely tocause the processor to utilize, per volume unit of measurement data, amuch larger number of computation cycles.

In one embodiment, processing levels can indicate to a processor whatmodels and/or what features of data are to be utilized by the processorto process affective response measurements. For example, a firstprocessing level may be used for basic processing of the data, andutilize models that involve a small subset of the data's features (e.g.,a small set of important features). A second processing level may beutilized for comprehensive processing and involve the full set offeatures that is available. Optionally, the processor receivesmeasurements of affective response that include data that comes frommultiple types of sources; for instance, data collected from a pluralityof sensors. In such cases, the processing level may indicate which ofthe sources to use. For example, in one case, measurements of a userinclude EEG, GSR and heart rate values; a first processing level mayindicate to process essentially only the heart rate data, while a secondprocessing level indicates to the processor to process all threesources. Thus, per volume unit of measurement data, a processoroperating at the first processing level utilizes significantly lesscomputation cycles compared to the processor operating at the secondprocessing level, since in the latter case, the processor performs manymore operations on the data.

In one embodiment, the processing level may indicate to the processorwhether to perform certain manipulations on data of affective responsemeasurements. For example, a first processing level may indicate to theprocessor that the data should not be encrypted and/or compressed, whilea second processing level may indicate to the processor that the datashould be encrypted and/or compressed. Thus, a processor processing thedata while operating at the first processing level is likely to utilizesignificantly less computation cycles, compared to when it operates atthe second processing level.

In some embodiments, a controller, such as the controller 175 or thecontroller 195 selects a measuring rate for a device used to measureaffective response based on an indication provided by a semanticanalyzer and/or a content ERA. Optionally, the controller selects themeasuring rate from amongst a plurality of measuring rates, where atleast some of the measuring rates differ with respect to the amount ofmeasurements the device is configured to take.

In one example, the semantic analyzer provides an indication that isindicative of whether a predetermined threshold was reached by a valuerelated to a prediction of emotional response to a segment of content.Depending on whether the threshold was reached, and/or the differencebetween the value and the threshold, different measuring rates may beselected possibly having power consumption rates, sampling frequencies,and/or dimensionality of measurement results.

In another example, the semantic analyzer provides an indication thatincludes a value that is indicative of a level of importance that may beattributed to the need to measure the user's affective response to thecontent. For instance, the indication may include a predicted interestlevel of a user in content, and the controller selects a measuring rate,such as a resolution of images captured by a camera, according to theinterest level. In this example, when the user is expected to beinterested in content, it is important to take detailed measurement ofthe user's affective response; so for instance, the measuring rate of acamera used to capture images of the user could be set to a higherresolution when a user is expected to be interested.

In one embodiment, when the controller selects a measuring rate, ittakes into consideration additional information that might not beconveyed in an indication received from the semantic analyzer and/orcontent ERA. For example, the controller of a device that is a batteryoperated sensor might take into consideration the current charge in thebattery used by the sensor, when choosing a measuring rate to operatethe device at.

In some embodiments, the controller selects a measuring rate for adevice, which causes the device to take fewer measurements, compared tothe measurements it takes when operating at one or more other measuringrates. For example, when configured to operate at a selected measurementrate, per unit of measurement time, the device takes a significantlylower number of measurements, produces a significantly lower amount ofmeasurement data, processes a significantly lower amount of data,transmits a significantly lower amount of data, and/or consumes asignificantly less power. Optionally, by “significantly lower” it ismeant at least 50% less. There are different possibilities in which ameasuring rate selected by the controller can cause the device tooperate in a way that reduces the measurements the device takes.

In one embodiment, a device may take fewer measurements by beingconfigured to operate at a measuring rate of substantially zero (andthus essentially the device does not measure the user). For example, thedevice, or certain components belonging to the device, may besubstantially turned off or operating in a standby mode.

In one embodiment, a device takes fewer measurements by being configuredto generate less data per unit of measurement time. For example,parameters such as resolution or frame rate (e.g., for a camera), or thenumber of channels to measure (e.g., for EEG), can be set, such that perunit of measurement, the device generates less data.

In another embodiment, a device may take fewer measurements by operatingat a measuring rate in which measurement data produced by the device isof a lower quality; thus, it may generate both fewer measurements, sincenoise may be filtered out, and/or fewer useful measurements. Forexample, a device such as a physiological sensor (e.g., an EEG sensor ora GSR sensor), can be configured to draw less power. This can cause themeasurements made by the device to be noisier, e.g., they may containmore random fluctuations and artifacts so the data might contain lessuseful measurement information.

In yet another embodiment, a device may be considered to take fewermeasurements by processing and/or transmitting less data. For example,while a sensor belonging to the device may measure the user at a normalrate, the device may be configured to process less data and/or transmitless data, thus effectively reducing the measuring rate of the device,as far as other components in the system may be concerned.

While the above embodiments described in the general context of programcomponents that execute in conjunction with an application program thatruns on an operating system on a computer, which may be a personalcomputer, those skilled in the art will recognize that aspects may alsobe implemented in combination with other program components. Programcomponents may include routines, programs, modules, data structures, andother types of structures that perform particular tasks or implementparticular abstract data types. Moreover, the embodiments may bepracticed with other computer system configurations, including hand-helddevices, multiprocessor systems, microprocessor-based or programmableconsumer electronics, minicomputers, mainframe computers, and comparablecomputing devices. The embodiments may also be practiced in adistributed computing environment where tasks are performed by remoteprocessing devices that are linked through a communications network. Ina distributed computing environment, program components may be locatedin both local and remote memory storage devices.

Embodiments may be implemented as a computer-implemented process, acomputing system, or as an article of manufacture, such as a computerprogram product or computer readable media. The computer program productmay be a computer storage medium readable by a computer system andencoding a computer program that comprises instructions for causing acomputer or computing system to perform example processes. Thecomputer-readable storage medium can for example be implemented via oneor more of a volatile computer memory, a non-volatile memory, a harddrive, a flash drive, a disk, a compact disk, and/or comparable media.

Throughout this specification, references are made to services. Aservice as used herein describes any networked/on line applications thatmay receive a user's personal information as part of its regularoperations and process/store/forward that information. Such applicationsmay be executed on a single computing device, on multiple computingdevices in a distributed manner, and so on. Embodiments may also beimplemented in a hosted service executed over a plurality of servers orcomparable systems. The term “server” generally refers to a computingdevice executing one or more software programs typically in a networkedenvironment. However, a server may also be implemented as a virtualserver (software programs) executed on one or more computing devicesviewed as a server on the network. Moreover, embodiments are not limitedto personal data. Systems for handling preferences and policies may beimplemented in systems for right management and/or usage control usingthe principles described above.

Herein, a predetermined value, such as a predetermined confidence levelor a predetermined threshold, is a fixed value and/or a value determinedany time before performing a calculation that compares its result withthe predetermined value. A value is also considered to be apredetermined value when the logic used to determine a threshold isknown before start calculating the threshold.

In this description, references to “one embodiment” mean that thefeature being referred to may be included in at least one embodiment ofthe invention. Moreover, separate references to “one embodiment” or“some embodiments” in this description do not necessarily refer to thesame embodiment.

The embodiments of the invention may include any variety of combinationsand/or integrations of the features of the embodiments described herein.Although some embodiments may depict serial operations, the embodimentsmay perform certain operations in parallel and/or in different ordersfrom those depicted. Moreover, the use of repeated reference numeralsand/or letters in the text and/or drawings is for the purpose ofsimplicity and clarity and does not in itself dictate a relationshipbetween the various embodiments and/or configurations discussed. Theembodiments are not limited in their applications to the details of theorder or sequence of steps of operation of methods, or to details ofimplementation of devices, set in the description, drawings, orexamples. Moreover, individual blocks illustrated in the figures may befunctional in nature and do not necessarily correspond to discretehardware elements.

While the methods disclosed herein have been described and shown withreference to particular steps performed in a particular order, it isunderstood that these steps may be combined, sub-divided, or reorderedto form an equivalent method without departing from the teachings of theembodiments. Accordingly, unless specifically indicated herein, theorder and grouping of the steps is not a limitation of the embodiments.Furthermore, methods and mechanisms of the embodiments will sometimes bedescribed in singular form for clarity. However, some embodiments mayinclude multiple iterations of a method or multiple instantiations of amechanism unless noted otherwise. For example, when an interface isdisclosed in one embodiment, the scope of the embodiment is intended toalso cover the use of multiple interfaces. Certain features of theembodiments, which may have been, for clarity, described in the contextof separate embodiments, may also be provided in various combinations ina single embodiment. Conversely, various features of the embodiments,which may have been, for brevity, described in the context of a singleembodiment, may also be provided separately or in any suitablesub-combination. Embodiments described in conjunction with specificexamples are presented by way of example, and not limitation. Moreover,it is evident that many alternatives, modifications, and variations willbe apparent to those skilled in the art. It is to be understood thatother embodiments may be utilized and structural changes may be madewithout departing from the scope of the embodiments. Accordingly, it isintended to embrace all such alternatives, modifications, and variationsthat fall within the spirit and scope of the appended claims and theirequivalents.

What is claimed is:
 1. A controller configured to utilize eye trackingdata to reduce power consumption of a device operative to measureaffective response, comprising: a first interface configured to receivea first tag that indicates a first duration during which to measureaffective response to a first segment of content by operating the devicein a first mode of operation; a second interface configured to receivefirst eye tracking data of a user acquired in temporal vicinity of thefirst duration; and a processor configured to: analyze the first eyetracking data, determine that a gaze-based attention level to the firstsegment reaches a first predetermined threshold, and to select the firstmode of operation for operating the device, during the first duration,to measure affective response of the user to the first segment; thefirst interface is further configured to receive a second tag thatindicates a second duration during which to measure affective responseto a second segment of content by operating the device in a first modeof operation; the second interface is further configured to receivesecond eye tracking data of the user acquired in temporal vicinity ofthe second duration; the processor is further configured to: analyze thesecond eye tracking data, determine that a gaze-based attention level tothe second segment does not reach a second predetermined threshold, andto select a second mode of operation for operating the device, duringthe second duration, to measure affective response of the user to thesecond segment; wherein power consumption of the device when operatingin the first mode of operation is significantly higher than the powerconsumption of the device when operating in the second mode ofoperation.
 2. The controller of claim 1, wherein the first and secondtags further comprise representations of expected emotional responses tothe segments.
 3. The controller of claim 2, further comprising atraining module configured to receive the first tag and measurementstaken by the device during the first duration, and to train an emotionalresponse model utilizing the first tag and the measurements.
 4. Thecontroller of claim 1, wherein the first tag depends on informationconveyed by the first segment, and does not depend mainly on behavior ofthe user while being exposed to the first segment.
 5. The controller ofclaim 1, wherein the first and second eye tracking data are acquiredutilizing an image capturing device, and the device is a battery poweredsensor that measures a physiological signal and is coupled to the user.6. The controller of claim 1, wherein the device is a battery poweredsensor that is coupled to the user and measures a physiological signal.7. The controller of claim 1, wherein the first and second eye trackingdata are collected substantially independently of whether the user isexposed to segments of content identified by tags; and the device is abattery powered sensor that is mainly used to measure affective responseof the user while exposed to segments identified by tags.
 8. Thecontroller of claim 1, wherein the first and second segments and thefirst and second tags are generated by an interactive computer gamehaving an element whose actions in the game are at least partiallycontrolled by the user.
 9. The controller of claim 1, wherein the powerconsumption of the device when operating in the second mode of operationis essentially zero.
 10. A method for utilizing eye tracking data toreduce power consumption of a device used to measure affective response,comprising: receiving a first tag that indicates a first duration duringwhich to measure affective response to a first segment of content byoperating the device in a first mode of operation; receiving a firstindication, derived from first eye tracking data of a user acquired intemporal vicinity of the first duration, indicating that a gaze-basedattention level to the first segment reaches a first predeterminedthreshold; configuring the device to operate in the first mode ofoperation to measure affective response of the user to the firstsegment; receiving a second tag that indicates a second duration duringwhich to measure affective response to a second segment of content byoperating the device in the first mode of operation; receiving a secondindication, derived from second eye tracking data of the user acquiredin temporal vicinity of the second duration, indicating that agaze-based attention level to the second segment does not reach a secondpredetermined threshold; configuring the device to operate in a secondmode of operation to measure affective response of the user to thesecond segment; wherein power consumption of the device when operatingin the first mode of operation is significantly higher than the powerconsumption of the device when operating in the second mode ofoperation.
 11. The method of claim 10, wherein the first tag furthercomprises a representation of expected emotional response to beingexposed to the first segment.
 12. The method of claim 11, furthercomprising forwarding, to a module that trains an emotional responsemodel, the first tag and measurements of the affective response of theuser to being exposed to the first segment.
 13. The method of claim 10,wherein the first tag depends on information conveyed by the firstsegment, and does not depend mainly on behavior of the user while beingexposed to the first segment.
 14. The method of claim 10, wherein thefirst and second eye tracking data are acquired utilizing an imagecapturing device and the device is a battery powered sensor thatmeasures a physiological signal and is coupled to the user.
 15. Themethod of claim 10, wherein the device is a battery powered sensor thatis coupled to the user, and measures a physiological signal.
 16. Themethod of claim 10, wherein the first and second eye tracking data arecollected substantially independently of whether the user is exposed tosegments of content identified by tags; and the device is a batterypowered sensor that is mainly used to measure affective response of theuser while exposed to segments identified by tags.
 17. The method ofclaim 10, further comprising generating the first segment and the firsttag by an interactive computer game having an element whose actions inthe game is at least partially controlled by the user; wherein the firsttag corresponds to a certain event occurring in the first segment.
 18. Acontroller configured to utilize eye tracking data to reduce powerconsumption of a device operative to measure affective response,comprising: a first interface configured to receive a first tag thatindicates a first duration during which to measure affective response toa first segment of content by operating the device in a first mode ofoperation; wherein the device is battery powered and measures aphysiological signal; a second interface configured to receive first eyetracking data of a user acquired in temporal vicinity of the firstduration; and a processor configured to analyze the first eye trackingdata and determine that a gaze-based attention level to the firstsegment reaches a first predetermined threshold, and to select the firstmode of operation for operating the device, during the first duration,to measure affective response of the user to the first segment; thefirst interface is further configured to receive a second tag thatindicates a second duration during which to measure affective responseto a second segment of content by operating the device in a first modeof operation; the second interface is further configured to receivesecond eye tracking data of the user acquired in temporal vicinity ofthe second duration; the processor is further configured to analyze thesecond eye tracking data and determine that a gaze-based attention levelto the second segment does not reach a second predetermined threshold,and to select a second mode of operation for operating the device,during the second duration, to measure affective response of the user tothe second segment; wherein power consumption of the device whenoperating in the first mode of operation is significantly higher thanthe power consumption of the device when operating in the second mode ofoperation.
 19. The controller of claim 18, further comprising a trainingmodule configured to receive the first tag and measurements taken by thedevice during the first duration, and to train an emotional responsemodel utilizing the first tag and the measurements; wherein the firsttag comprises a representation of an expected emotional response to thefirst segment.
 20. The controller of claim 18, wherein the first andsecond tags depend on information conveyed by the first and secondsegments, and do not depend mainly on behavior of the user while beingexposed to the first and second segments.